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ABSTRACT 


This  thesis  presents  an  application  of  Multidimensional  Scaling 
(MEG)  used  in  the  prioritization  of  ASM  decision  functions  in  the 
S-3A.  The  ASW  decision  space  was  divided  into  14  discrete  decision 
functions  for  purposes  of  this  analysis.  The  problem  of  developing  a 
prioritization  methodology  was  approached  from  two  independent 
directions. 

First,  an  unconstrained  sorting  task  was  preformed  to  provide 
input  to  a  Multidimensional  Scaling  algorithm.  The  result  of  this 
analysis  provided  a  three  dimensional  representation  of  the  decision 
space  with  dimensional  interpretation.  Second,  a  series  of  ranking 
tasks  were  preformed  to  provide  input  to  an  Unfolding  Analysis 
algorithm.  The  Generalized  Distance  Model  was  selected  as  the  model 
most  representative  of  the  ranking  data. 

The  decision  function  coordinates  for  the  MDS  algorithm  and  the 
decision  function  coefficients  for  the  Unfolding  Analysis  algorithm 
were  combined  in  a  regression-like  equation  to  provide  a 
prioritization  methodology  for  the  14  decision  functions  of  the  S-3A 
ASM  decision  space. 
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I.  INTRODUCTION 


A.  HISTORY 

Decision  making  and  the  process  by  which  decisions  are 
made  in  the  complex  world  of  Antisubmarine  Warfare  (ASW)  has 
long  confounded  the  designer  as  well  as  the  operator  of 
modern  sophisticated  weapons  systems.  Operator  [Zachary 
1980a],  software,  and  hardware  [NAVAIR  1975]  limitations 
have  been  cited  as  causes  of  less  than  optimal  performance 
in  the  modern  ASW  mission. 

Antisubmarine  Warfare  has  traditionally  been  thought  of 
as  a  half  science  -  half  art  method  of  locating  hostile 
submerged  vessels.  Originally,  ASW  was  conducted  solely  by 
war  ships  using  active  ranging  equipment  where  tactical 
'guessing'  filled  the  information  gaps  left  by  onboard 
sensors.  As  the  missions  of  s  •‘■'marines  became  more 
diversified,  air  and  subsurface  weapons  platforms  were 
developed  to  augment  existing  surface  weapons  systems.  With 
the  advent  of  computers,  the  information  gaps  of  past 
sensors  narrowed  considerably.  However,  while  the  operator 
received  more  accurate,  real  time  information,  the  quantity 
of  this  new  information  could,  at  times,  overwhelm  even  a 
seasoned  operator.  To  enhance  the  operator's  ability  to 
perform  the  ASW  mission,  systems  were  developed  to  integrate 
the  large  quantity  of  information,  filter  the  'noise'  from 
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the  needed  inputs,  and  present  the  information  in  an 
orderly,  useful  manner.  Such  systems  were  implemented  for 
surface,  subsurface,  and  airborne  weapons  systems. 

B.  ASW  MISSION 

All  ASW  weapons  systems  -  airborne,  surface,  and 
subsurface  -  share  four  basic  phases  of  mission 
accomplishment:  search,  localization,  track,  and  attack.  The 
first  phase,  search,  involves  the  planning  and 
implementation  of  strategies  (sensor  placement,  areas  of 
search,  initial  tactics)  to  gain  initial  detection  of 
potential  threat.  In  the  second  phase,  the  potential  threat 
is  localized  through  the  use  of  additional  sensors  and 
detection  information.  After  localization  of  the  potential 
threat,  the  third  phase,  tracking  <is  accomplished  by  the 
integration  of  sensor  information  and  tactics.  In  peacetime, 
tracking  is  the  mission  priority,  while  in  wartime,  the 
fourth  phase,  attacking,  receives  a  higher  priority.  These 
phases  should  be  followed  systematically  to  assure,  in  time 
of  peace,  constant  position  information  on  potential 
threats,  or  in  time  of  war,  destruction  of  hostile 
submarines. 

Throughout  these  four  phases,  ASW  personnel  are  engaged 
in  decision  making  tasks  of  various  degrees  of  complexity 
and  difficulty.  Decision  aids  could  be  of  great  use  in 
performing  these  tasks.  Antisubmarine  Warfare  is  a  dynamic 
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environment  in  which  one  can  not  train  for  all  possible 

conditions.  Accordingly,  emphasis  is  placed  on  software,  in 
the  form  of  decision  aids,  to  assist  the  operator  in  the 
performance  of  the  ASW  mission.  However,  decision  aids  have 
traditionally  been  based  on  designer  subjectivity,  random 
fleet  inputs,  and  in  response  to  a  new,  immediate  threat. 
Regretably,  fleet  ASW  personnel  have  had  little  impact  on 
the  design  and  implementation  of  ASW  decision  aids.  A 

prioritization  of  ASW  decision  areas  in  which  decision  aids 
could  be  of  benefit  would  directly  impact  on  all  ASW  assets. 

Because  of  the  multitude  of  ASW  platform  types  in  the 
U.S.  Navy,  the  scope  of  this  thesis  was  limited  to  one 

platform  in  the  accomplishment  of  the  ASW  missic  The 
criterion  for  selection  were  (1)  an  operator  whose  primary 
duty  is  ASW,  (2)  a  'state-of-the-art'  weapons  system 
(hardware  and  software)  ,  and  (3)  ease  of  data  collection. 
The  S-3A  Viking  aircraft  fulfilled  these  criteria  and  was 
selected  as  the  wearpons  platform  of  interest. 

C.  PLATFORM  OF  INTERESTS 

The  S-3A  Viking,  a  carrier  based,  twin  engine  jet 

aircraft,  with  an  onboard  digital  computer,  sophisticated 
avionics  and  crew  of  four  is  considered  to  be  the  Carrier 
Battle  Group's  first  line  of  defense  from  hostile 
submarines.  Although  the  S-3A  has  other  missions,  only  the 
ASW  mission  will  be  addressed  in  this  thesis.  The  crew  of 
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the  S-3A  consists  of:  Pilot,  who  is  responsible  for  safety 
of  flight;  the  Copilot,  who  operates  the  nonacoustic  sensors 
(Radar,  Flir,  ESM,  and  MAD);  the  Acoustic  Sensor  Operator 
(SENSO) ,  who  is  responsible  for  passive  and  active  acoustic 
surveillance;  and  the  Tactical  Coordinator  (TACCO) ,  who  is 
normally  the  tactical  mission  commander  and  is  responsible 
for  the  prosecution  of  subsurface  contacts.  Because  the 
TACCO  receives  information  from  all  other  members  of  the 
crew  as  well  as  the  onboard  General  Purpose  Digital  Computer 
(GPDC)  for  his  tactical  decision  making,  he  will  be  of 
primary  interest  in  establishing  priority  for  decision  aids. 
For  definitive  purposes,  a  decision  aid  is  any  methodology, 
algorithm,  queue,  or  filtering  mechanism  that  enhances  an 
operator's  performance  by  eliminating  unwanted,  extranious 
information  and  presents  critical  information  in  an  orderly, 
streamline  fashion  to  expedite  a  needed  response. 


D.  PURPOSE 

The  purpose  of  this  work  was  to  examine  ASW  decision 
making  in  the  S-3A  Viking  through  the  use  of  mathematical 
modeling  techniques  and  to  establish  a  prioritization 
technique  for  the  development  of  decision  aids  to  assist 
decision  making  in  the  S-3A. 

E.  ORGANIZATION  OF  THIS  THESIS 

The  main  body  of  this  thesis  addresses  the  results  of 
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Multidimensional  Scaling  techniques  and  the  resulting 
applications  to  decision  aids.  Chapter  2  discusses  the 
advantages  and  disadvantages  of  various  scaling  methods 
classified  under  the  general  heading  of  Multidimensional 
Scaling.  The  methods  evaluated  included  pairwise  comparison, 
unconstrained  sorting  (Q-Sort)  ,  and  the  triad  method. 
Chapter  3  discusses  the  data  collection.  Chapter  4  contains 
the  data  analysis  based  on  the  unconstrained  sorting 
technique  with  emphasis  on  Unfolding  Analysis.  Finally, 
Chapter  5  discusses  the  results,  conclusions,  and 
recommendations . 
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II.  MULTIDIMENSIONAL  SCALING  TECHNIQUES 


A.  OVERVIEW 

ASW  decision  making,  because  o £  the  complexity  and 
quantity  of  information  flow,  lends  itself  to  few 
mathematical  modeling  techniques.  ASW  decisions,  while  based 
on  a  logical  flow  of  events,  are  subjective  in  nature 
yielding  ordinal  data  points  at  best. 

Pew  modeling  techniques  can  use  data  of  less  than 
interval  nature.  However,  a  family  of  modeling  techniques 
does  exist  that  use  nonmetric  (ordinal  data)  as  well  as 
metric  (interval  or  ratio  data)  inputs.  One  such  group  of 
techniques  is  Multidimensional  Scaling  (MDS) .  Before  an 
application  of  Multidimensional  Scaling  can  be  used  to 
examine  ASW  decision  making,  a  working  understanding  of  the 
technique  should  be  attained. 

Multidimensional  Scaling  is  the  term  loosely  used  to 
identify  a  large  group  of  varied  and  powerful  techniques  for 
the  analysis  of  data  normally  associated  with  the  behavioral 
sciences.  There  exist  two  main  purposes  for  utilizing  these 
techniques  -  (1)  to  detect  any  pattern,  structure,  or 
relationship  that  may  be  hidden  in  a  matrix  of  empirical 
data,  and  (2)  to  represent  that  structure  in  a  form  that  is 
intuitively  appealing  and  is  more  intelligible  to  the  human 
(i.e.  a  two  or  three  dimensional  graph  vice  a  five  or  six 


degree  dimensionalization)  .  The  stimuli  under  study  are 
represented  by  points  in  this  geometric  model  such  that  the 
significant  facets  of  the  data  points  (stimuli)  are  revealed 
in  their  relationship  among  other  stimuli  [Shepard  1972]. 

The  spatial  representation  resembles  more  traditional 
relationship  scales  such  as  temperature  or  time  in  that  it 
attempts  to  acquire  the  fundamental  properties  (structure) 
of  stimuli  under  study  solely  by  setting  them  into 
correspondence  with  positions  within  a  spatial  subset  of  N 
dimensionality.  Zt  differs  from  simple  unidimensional  (i.e. 
conjoint  measurement)  scales  in  that  in  order  to  utilize  all 
of  the  information  provided  by  the  data  or  capture  the  full 
complexity  of  the  stimuli,  the  data  points  may  assume 
positions  within  a  two,  or  three  dimensional  space  as  well 
[Kruskal  1978] . 

In  most  cases,  one  seeks  a  presentation  of  the  model  of 
the  lowest  possible  dimensionality  consistent  with  the 
problem  to  be  solved  and  the  data.  Obviously,  a  lower 
dimensional  model  would  be  easier  to  understand  intuitively 
and  easier  to  work  with,  for  it  represents  the  data  by  means 
of  a  smaller  number  of  parameters.  Minimizing  the  number  of 
parameters  also  yields  a  more  reliable  statistical  base  due 
to  larger  data  subsets.  However,  the  reduction  of 
dimensionality  is  not  an  end  unto  itself.  One  runs  the  risk 
of  damage  to  the  data  by  arbitrarily  reducing  the 
dimensionality  of  the  model.  If  the  relationship  among 
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stimuli  is  sufficiently  complex,  two  or  three  dimensions, 
then  the  model  better  'fits'  the  empirical  data.  The 
goodness  of  fit  measurement  called  'stress'  is  the  parameter 
that  measures  the  deviation  of  the  empirical  data  from  the 
model.  There  exist  a  gray  area  of  trade-offs  between  easy 
visualization  and  the  use  of  all  data  information.  It  is  the 
analyst's  responsibility  to  execute  the  trade-off  after 
evaluating  the  criteria  for  analyzing  the  model. 

Multidimensional  Scaling  utilizes  the  same  basic 
concepts  as  Factor  Analysis.  The  high  dimensionality, 
usually  associated  with  Factor  Analysis,  is  caused  by  the 
rigid  assumption  of  linearity  among  stimuli.  This  assumption 
is  relaxed  in  the  model  used  for  Multideminsional  Scaling, 
allowing  (normally)  a  two  or  three  dimensional 
representation.  Essentially,  Multidimensional  Scaling 
provides  greater  stimuli  resolution  in  a  more  readily 
visualizable  model  than  Factor  Analysis. 

Multidimensional  Scaling  uses  similiarities  or 
dissimiliarities  among  stimuli  to  define  the  relationships 
that  exist  in  each  possible  dimension.  Several  subsets  of 
multidimensional  scales  exist  in  various  forms  for  different 
appi lcations .  The  subsets  include  paired  and  triad 
comparisons,  unconstrained  sorting,  conjoint  measurement, 
delphi  method,  and  multiple  attribute  utility  models. 
However,  the  major  subsets  considered  in  this  thesis  were 
limited  to  pairwise  comparison,  unconstrained  sorting,  and 


triad  comparison  because  of  external  constraints  placed  on 
the  data  collection.  The  constraints  were  the  short  periods 
of  time  available  for  data  collection  and  the 
inaccessability  of  subjects  after  the  initial  interview.  The 
constraints  will  be  discussed  throughly  in  Chapter  III. 
Because  of  the  multiple  passes  required  through  the  data  and 
subject  interaction  required,  the  techniques  of  multiple 
attribute  utility,  delphi  and  conjoint  measurement  were 
considered  less  desirable  than  the  selected  techniques. 

The  measure  of  effectiveness  for  the  pairwise 
comparison,  unconstrained  sort,  and  triad  comparison  is  how 
well  the  model  or  technique  represents  the  empirical  data. 
This  measure  of  effectiveness  is  called  'stress'  [Kruskal 
1978],  Stress  is  the  goodness  of  fit  measurement  for  the 
model  used  to  determine  the  number  of  dimensions  considered, 
balanced  by  the  amount  of  deviation  from  the  data.  Stress 
measures  the  degree  of  departure  from  the  assumption  that 
there  is  a  monotonic  relationship  between  the  non-metric 
dissimiliarity  measures  of  the  data  and  the  metric  distance 
measures  of  the  representational  structure  [Burton  1968). 

B.  PAIRWISE  COMPARISON 

The  most  common  technique  is  the  pairwise  comparison  of 
stimuli  with  a  ranking  or  weighting  between  the  two  [Kruskal 
1978].  The  pairwise  comparison  examines  N  different  stimuli 
in  pairs,  assigning  relative  ranks  as  to  the  degree  of 
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similiarity  or  dissimiliarity  of  the  two  stimuli  being 
compared.  The  comparisons  are  used  to  form  matrices  of 
ordinal  preference  data.  The  pair-wise  data  is  examined  for 
hidden  relationships  among  three  or  more  stimuli  that  would 
not  normally  come  to  light  in  pair-wise  preference  mapping. 
These  relationships  form  the  dimensionality  of  the  model  and 
point  to  areas  of  greatest  (or  least)  interest. 

While  the  pairwise  technique  utilizes  the  greatest 
amount  of  information  from  a  given  set  of  data,  several 
limitations  become  apparent  in  the  use  of  this  comparison 
method.  For  N  stimuli,  the  number  of  pair-wise  comparisons 
are  N  (N  -  l)/2.  For  all  but  a  small  number  of  comparisons 
(N  <  10) ,  the  number  becomes  unmanageable.  The  complexity 
increases  with  larger  subject  samples.  Additionally,  the 
tester  normally  has  a  finite  time  limit  and  even  a  small 
sample  (N  ■  8,  subjects  »  10)  requires  a  large  amount  of 
time. 

C.  UNCONSTRAINED  SORTING 

The  unconstrained  sorting  task  [Burton  1975]  requires 
subjects  to  sort  stimuli  into  groups.  The  sorting  into 
groups  represents  a  partitioning  of  the  stimuli  set. 
Normally,  the  stimuli  set  consist  of  the  stimuli  represented 
in  a  medium  that  is  easily  sorted  (such  as  3  X  5  cards)  and 
test  subjects  are  asked  to  sort  the  stimuli  into  groups.  The 
instructions  for  the  formation  of  these  groups  require  the 
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subjects  to  sort  the  stimuli  that  are  'similar  in  meaning' 
or  'belong  together'  into  the  same  group.  The  subjective 
interpretation  of  the  sorting  instructions  by  each  member  is 
the  basis  for  an  unconstrained  sort.  The  subsets  (groups)  of 
stimuli  can  have  any  number  of  members.  A  single  stimulus  in 
a  group  represents  no  similarity  to  any  other  stimulus  and 
all  stimuli  in  a  group  represent  total  similarity  among 
stimuli.  Test  subjects  who  have  few  subgroups  of  stimuli  are 
called  'lumpers'  and  subjects  who  make  many  distinctions 
among  stimuli  are  referred  to  as  'slitters'.  Burton's  study 
of  Q-Sort  data  [1975]  suggest  that  among  Multidimensional 
Scaling  techniques,  the  unconstrained  sorting  task  is  the 
most  flexible  in  terms  of  testing  procedures. 

D.  TRIAD  TEST 

The  triad  test  is  similar  in  procedure  to  the  paired 
comparisons  discussed  earlier  except  stimuli  are  compared 
three  vice  two  at  a  time.  The  triads  test  may  be 
administered  in  two  different  ways.  The  first  way  requires 
the  subject  to  choose  the  most  different  stimulus  from  among 
the  three  stimuli  presented.  The  second  version  of  the  test 
asks  the  subject  to  pick  the  most  different  stimulus  (as 
before)  and  also  to  pick  the  pair  of  stimuli  which  are  most 
different  (one  from  another)  [Burton  1975]. 

There  are  (  3  )  or  N  (N  -  1)  (N  -  2)/6  triads  for  N 
stimuli.  These  triads  are  used  to  form  matrices  of  ordinal 
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preference  data.  This  ordinal  data  is  monotonically 
transformed  to  a  metric  form  that  can  be  examined  for 
relationship  or  structures  that  are  not  apparent  at  first 
inspection. 

E.  COMPARISON  OF  TECHNIQUES 

Although  the  pairwise  comparison  was  the  most  common 
test  found  in  the  literature  review  [Shepard  1972]  ,  [Kruskal 
1978] ,  the  unconstrained  sort  and  triads  test  revealed 
several  interesting  advantages  for  the  implementation  of 
either  technique.  The  advantages  and  disadvantages  of  each 
technique  are  discussed  in  this  section. 

All  three  techniques  share  the  advantages  of  being  a 
simple  task  to  understand  and  all  can  be  presented  either 
verbally  or  in  written  form.  At  this  point,  the  paired 
comparision  and  triads  test  begin  to  differ  from  the 
constrained  sort.  For  N  stimuli  and  M  subjects,  the  number 
of  pairs  presented  by  the  tester  is  (  )M  or 
N(N  -  1) M/2  and  the  number  of  triads  presented  equal  (  ^  )M 
or  N (N  -  1 ) (N  -  2) M/6.  Intuitively,  one  can  see  the 

advantage  of  the  triads  test  in  the  sheer  number  of  stimuli 
groups.  However,  for  N  £  10  and  M  >  10,  either  test  soon 
becomes  unmanageable  in  terms  of  time  necessary  to  give  the 
test . 

On  the  other  hand,  the  unconstrained  sorting  technique 
has  one  clear  advantage  :  time.  The  unconstrained  sort  can 
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be  given  to  several  subjects  at  once  and  recent  research  by 
Burton  [1975]  suggests  that  most  people  can  do  a  single 
sorting  of  sixty(60)  stimuli  in  fifteen  to  thirty  minutes. 
Additionally,  experiments  by  Miller  [1974]  indicate  that 
triads  test  and  sorting  task  uncover  the  same  structure 
within  sorting  decisions. 

F.  SELECTION  OF  TECHNIQUE 

As  discussed  in  the  preceeding  sections,  each  of  the 
three  Multidimensional  Scaling  techniques  can  be  applied  to 
a  great  number  of  problem  areas  with  varying  degrees  of 
success.  An  examination  of  the  problem  area  suggested  that 
one  technique  was  superior  to  the  other  two. 

The  problem  concerned  the  ASW  operational  decision 
making  environment,  specifically  the  S-3A  aircraft.  Because 
of  the  dynamics  of  the  S-3A  environment  ashore  (Weapons 
Systems  Trainers,  ground  schools,  and  training  flights) ,  it 
was  essential  to  interview  as  many  experienced  TACCOs  as 
possible  in  the  shortest  amount  of  time  so  as  not  to 
interfere  with  training  and  operational  commitments. 
Consequently,  with  time  as  the  major  constraint,  the 
unconstrained  sorting  technique  was  selected. 
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APPLICATION 


A.  PROBLEM 

The  development  of  air  ASW  decision  aids  requires  an 
understanding  of  the  complexities  and  the  interrelationships 
among  decisions  of  the  ASW  mission.  The  measure  of 
importance  with  respect  to  ASW  decision  making  is  a 
non-quant i f i able  variable  that  directly  impacts  on 
priorities  among  decisions.  Because  of  the  subjective  nature 
of  ASW  decision  making,  Multidimensional  Scaling,  in 
general,  and  the  Unconstrained  Sorting  Task,  specifically, 
lend  themselves  favorably  to  the  prioritization  of  the  ASW 
decision  space.  An  examination  of  the  specific  S-3A  ASW 
decision  making  process  was  helpful  as  an  overview  to  the 
problem. 

As  mentioned  in  the  introduction,  the  air  ASW  mission 
was  divided,  into  four  general  phases:  (1)  search,  (2) 
localization,  (3)  track,  and  (4)  attack.  While  all  of  these 
areas  are  important,  the  number  of  stimuli  (phases) 
concerning  the  mission  was  too  small  to  yield  any 
appreciable  information  about  the  decision  function.  An 
improvement  over  the  four  phase  model  was  the  repartitioning 
by  Zachary  [1980a]  of  the  air  ASW  decision  space  into  six 
specific  decision  situations.  The  six  specific  decision 
situations  were: 
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1)  On-station  search 

2)  Contact  classification/verification 

3)  Localization 

4)  Surveillance  tracking 

5)  Attack  planning 

6)  Lost  contact  requisition 

The  six  decision  situations  model,  while  an  improvement 
over  the  four  phase  model,  did  not  remedy  the  problem  of 
limited  information  used  as  input  to  Multidimensional 
Scaling  models.  The  six  'phase'  model  was  further  evaluated 
by  Zachary  [1980b]  and  found  to  have  varying  numbers  of 
constituent  decision  functions  associated  with  each  phase. 
These  constituent  decision  functions  were  not  considered  to 
be  confined  to  only  one  decision  situation  but  rather  each 
decision  function  could  be  used  in  any  or  all  combinations 
of  the  decision  situation  (phases)  to  which  that  decision 
function  was  considered  applicable.  Table  I  shows  the 
decision  function  composition  of  the  six  air  ASW  decision 
situations. 

From  a  careful  examination  of  the  decision  situations  in 
Table  I,  fourteen  (14)  distinct  decision  functions  emerged 
as  necessary  to  completely  describe  the  air  ASW  decision 
space  within  the  context  of  this  thesis.  Consequently,  the 
fourteen  decision  functions  listed  in  Table  II  became  the 
stimulus  set  for  the  multidimensional  scaling  sampling.  The 
decision  functions  and  their  descriptions  are  listed  in 
Table  II. 
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Table  I 


Decision  Function  Composition  of  Six 
Air  ASW  Decision  Functions 


OECISION  SITUATION 

CONSTITUENT  DECISION  FUNCTIONS 

ON-STATION  SEARCH 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

EXTEND  SENSOR  PATTERN 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

CONTACT  CLASSIFICATION/VERIFICATION 

CLASSIFY  SIGNAL 

DETERMINE  SIGNAL  IS  A  VALID  CONTACT 

LOCALIZATION 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

GAIN  ATTACK  CRITERIA 

DETERMINE  TARGET  FIX 

COORDINATE  HAND-OFF 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

SURVEILLANCE  TRACKING 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

DETERMINE  TARGET  FIX 

COORDINATE  HAND-OFF 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

ATTACK  PLANNING 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

GAIN  ATTACK  CRITERIA 

DETERMINE  TARGET  FIX 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK 
CONSTRUCT  SENSOR  MONITORING  PATTERN 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH  POSITION 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

LOST  CONTACT  REACQUISITION 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

COORDINATE  HAND-OFF 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 
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Table  II.  Description  of  the  Fourteen  Decision  Functions 

-  ADJUST  PATTERN  TO  SENSOR  FAILURE  (AP) 
replacement  of  faulty  sensors  or  adjustment  of 
aircraft  track  due  to  requipment/sensor  failure. 

-  EXTEND  EXISTING  SENSOR  PATTERN  (EP)  -  determine 
orientation  and  settings  of  sensors  to  be  added  to 
existing  pattern  with  no  contact. 

-  ANTICIPATE  TARGET  MOVEMENT  (AM)  -  predict  future 
location,  course,  speed,  and  depth  of  target  based 
on  intelligence  and  actual  tactical  situation. 

-  CONSTRUCT  SENSOR  MONITORING  PATTERN  (MP)  - 
determine  where  to  position  the  aircraft  to  obtain 
maximum  reception  from  sensors,  and  sequence  in 
which  to  monitor  deployed  (acoustic)  sensors  if 
number  of  sensors  exceeds  number  of  aircraft 
receivers . 

-  COORDINATE  HAND-OFF  (CH)  -  transmission  of  data 
to  relief  platform  for  continuation  of  prosecution. 

-  DETERMINE  SIGNAL  IS  A  VALID  CONTACT  (VC)  -  reduce 
false  alarm  rates  of  sensor  system  (e.g.,  clouds  on 
radar,  random  noise  on  MAD  or  acoustic). 

-  DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK  (DS)  - 
select  weapon  to  be  used  for  the  attack,  and  its 
optimum  weapon  settings  (e.g.,  search  depth, 
minimum/maximum  search  limit) . 

-  GAIN  ATTACK  CRITERIA  (AC)  -  interpret  sensor  data 
to  determine  when  target  is  localized  sufficently 
to  place  an  attack. 

-  DETERMINE  TARGET  FIX  (TF)-  use  incoming  sensor 
data  to  establish  the  location  of  the  target. 

-  CREATE  SENSOR  PATTERN  (CP)  -  determine  pattern, 
spacing,  orientation  and/or  utilization  for  all 
sensor  types  for  each  phase  of  the  mission. 

-  MANAGE  EQUIPMENT/STORES  TO  ACCOMODATE  PRESENT  AND 
FUTURE  NEEDS  (ME)  -  monitor  inventory  of  sensors 
and  equipment  status  for  application  for  future 
tactics  and  resources. 
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Table  II.  Continued 


-  CLASSIFY  SIGNAL  (CS)  -  determine  if  signal 
(primarily  acoustic)  may  be  originating  from  the 
target  of  interest. 

-  COMPENSATE  FOR  ACOUSTIC/ATMOSPHERIC  PROPAGATION 
CONDITIONS  (PC)  -  determine  the  adjustments  in 
sensor  spacing,  aircraft  track,  aircraft  altitude, 
etc.,  that  must  be  made  when  actual  (in  situation) 
atmospheric  and  bathythermal  conditions  are 
different  from  their  forecast. 

-  DETERMINE  AIRCRAFT  WEAPON  LAUNCH  POSITION  (LP)  - 
determine  factors  to  place  the  aircraft  in  an 
optimum  attack  position. 

In  this  analysis,  the  TACCOs  were  asked  to  perform  two 
psychometric  tasks  to  elicit  indicators  as  to  the  subjective 
nature  of  ASW  decision  making.  The  first  task  was  an 
unconstrained  sorting  of  the  stimulus  set  of  14  decision 
functions  that  provided  the  necessary  inputs  for  the 
Multidimensional  Scaling.  The  second  task  was  a  set  of  rank 
orderings  of  the  stimulus  set  that  provided  inputs  to  the 
unfolding  analysis  and  the  correlations  testing.  Both  task 
will  be  discussed  in  detail  in  the  following  sections. 

1.  Unconstrained  Sorting  Task 

The  technique  of  the  unconstrained  sorting,  also 
know  as  the  'Q-sort' ,  was  first  developed  by  Stephenson 
[1953].  The  technique  was  further  refined  by  Burton  [1968] 
and  applications  of  the  scaling  were  validated  by  Miller 
[1974],  In  this  task,  the  subject  is  asked  to  partition  the 
members  of  the  stimulus  set  into  an  arbitrary  number  of 
groups  of  unspecified  size.  The  criterion  for  sorting 
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usually  is  a  similarity  or  dissimiliarity  judgement  by  the 
subject  established  by  the  instructions  Eor  the  task.  There 
is  no  limit  on  the  number  of  stimuli  in  a  group  nor  is  there 
a  limit  on  the  number  of  groups. 

As  discussed  in  the  previous  chapter,  the 
unconstrained  sorting  task  is  only  one  of  many  psychometric 
methods  used  to  collect  data  for  Multidimensional  Scaling. 
Unconstrained  sorting  was  chosen  over  the  other  discussed 
techniques  for  two  reasons.  First,  the  unconstrained  sort 
allows  the  levels  of  discrimination  among  the  stimuli  by 
each  subject  to  be  controlled  for  explicitly  in  the 
processing  of  the  results.  Subjects  will  place  varying 
degrees  of  emphasis  on  the  underlying  structure  of  the 
interrelationships  among  stimuli.  The  algorithms  developed 
for  this  technique  in  the  processing  of  data  inputs  control 
for  between  subject  variance.  Subjects,  however,  vary  widely 
in  the  level  of  detail  perceived  to  be  necessary  for 
performance  of  the  sorting  task.  The  perceived  level  of 
detail  task  performance  can  differentiate  broad  distinctions 
that  find  all  stimuli  very  much  alike  or  very  different;  or 
from  very  few  distinctions  that  find  small  groups  of  stimuli 
similiar  but  in  different  degrees.  Depending  on  the  degree 
of  distinction  required  by  the  interviewer,  the  instructions 
must  be  expressed  so  as  to  focus  on  a  specific  level  of 
distinction  required  of  a  scaling  task.  This  is  the  primary 
reason  for  choosing  the  unconstrained  sorting  task. 
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Secondly,  the  unconstrained  sort  is  the  simplest, 
most  expeditious  method  of  collecting  data  for 
Multidimensional  Scaling.  As  discussed  in  the  previous 
chapter,  the  limited  amount  of  time  available  for  TACCO 
interviews  was  critical.  The  sorting  task  portion  of  the 
interview  took  less  than  30  minutes  for  each  TACCO,  favoring 
this  technique  over  more  time  consuming  methods. 

Thorough  investigation  of  Burton  [1968],  Miller 
[1974]  and  Zachary  [1980b]  have  shown  that  unconstrained 
sorting  produces  equivalent  results  to  the  other  MDS 
techniques  when  two  general  rules  of  thumb  are  obeyed.  The 
first  rule  is  that  the  number  of  stimuli  must  be  greater 
than  ten,  (S  >  10),  and  second,  there  must  be  at  least  twice 
as  many  subjects  (N)  performing  the  sort  as  there  are 
stimuli,  (N  2l  2S)  .  The  fourteen  decision  functions, 
previously  discussed,  meet  the  stimuli  number  requirement 
and  a  total  of  30  S-3A  TACCOs  interviewed  satisfied  the 
second  condition. 

Each  TACCO  interviewed  was  presented  the  sorting 
task  stimuli  verbally,  in  the  briefing,  and  in  writing,  in 
the  form  of  a  deck  of  3X5  inch  index  cards.  The  written  and 
the  verbal  presentations  were  identical  in  nature.  In  the 
center  of  each  stimulus  card  (in  bold  print)  was  the  name  of 
the  decision.  Following  the  name  was  a  brief  explanation  of 
the  decision  function.  To  identify  each  stimulus  card,  a  two 
letter  code  was  used  (in  the  upper  right  corner)  to  record 


the  sorting  response  as  well  as  the  ranking  (to  be  discussed 
later) .  Figure  1  represents  a  sample  decision  card  used  in 
the  interviews.  The  codes  were  mnemonic  in  nature,  derived 
from  the  keywords  of  the  decrision  functions.  Table  II  lists 
the  codes  and  the  decision  functions  used  in  the  data 
collection.  Mnemonic  codes  were  selected  instead  of  numeric 
or  alphabetic  codes  to  eliminate  any  bias  in  the  way  the 
cards  were  sorted  due  to  any  unintentional  implicit 
ordering . 


Figure  1.  Sample  Decision  Function  Card 


AM 


ANTICIPATE  TARGET  MOVEMENT 


predict  future  location,  course,  speed,  and 
depth  of  target  based  on  intelligence  and 
actual  tactical  situation 


2.  Ranking  Task 

The  second  psychometric  task  performed  by  the  TACCOs 
was  a  rank  ordering  task.  The  subjects  used  the  stimulus 
cards,  discussed  previously,  and  ranked  the  stimuli 
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according  to  four  different  criteria.  The  criteria  for  the 
four  rankings  were: 

1)  Importance  of  the  decision  in  the  ASW  mission 
with  an  attack  on  a  submarine. 

2)  Importance  of  the  decision  in  the  ASW  mission 
with  only  surveillance  of  a  submarine. 

3)  Urgency  of  the  decision,  and 

4)  TACCO  workload  during  the  decision. 

Rankings  1  and  2  provided  inputs  for  the  Unfolding 
Analysis  (discussed  in  Chapter  IV)  .  They  were  selected  for 
the  Unfolding  Analysis  because  they  use  the  same  criterion  - 
importance  of  the  decision  to  the  accomplishment  of  mission 
-  but  under  different  circumstances  -  attack  versus 
surveillance.  This  difference  was  used  as  a  reference.  The 
criterion  of  importance  was  defined  as  the  degree  to  which  a 
less  than  optimal  decision  would  adversely  impact  on  the 
achievement  of  the  mission  objective. 

Urgency,  the  third  ranking  criterion,  was  defined  as 
the  relative  speed  with  which  the  decision  had  to  be  made, 
at  a  decision  point.  The  fourth  ranking  criterion,  workload, 
was  defined  only  in  the  most  general  terms.  Each  TACCO  was 
instructed  to  consider  mental  as  well  as  physical  workload. 
These  last  two  rankings  were  used  to  provide  supplemental 
information  which  could  aid  in  the  interpretation  of  the  MDS 
solution. 
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B.  DATA  COLLECTION 


This  thesis  is  based  on  the  assumption  that  a 
priorization  of  the  S-3A  ASW  decision  space  must  include  the 
knowledge,  opinion,  and  intuition  of  experienced  S-3A  ASW 
decision  makers  (TACCOs) .  Multidimensional  Scaling  and 
Unfolding  Analysis,  discussed  in  the  data  analysis  section 
of  this  thesis,  provided  a  way  for  numerically  prioritizing 
the  fourteen  decision  functions  previously  discussed,  on  the 
basis  of  structured  but  subjective  inputs  from  S-3A  TACCOs. 

The  primary  source  of  experienced  S-3A  TACCO's  for  this 
analysis  was  VSWING  1,  NAS  Cecil  Field,  Jacksonville, 
Florida.  For  data  collection  purposes,  an  experienced  TACCO 
was  defined  as  an  individual  who  had  been  designated  an  S-3A 
TACCo  for  at  least  one  year  or  had  made  at  least  one 
deployment  as  a  qualified  S-3A  TACCO.  VSWING  1  was  selected 
as  the  sample  TACCO  population  primarily  because  of  the 
availability  of  experienced  TACCOs  during  the  limited  time 
frame  of  the  data  collection  effort. 

The  structure  of  the  Multidimensional  Scaling  interview 
consisted  of  three  general  parts:  1)  initial  briefing  and 
explanation,  2)sorting  task,  and  3)  ranking  task.  At  the 
conclusion  of  the  interview  an  open-ended  discussion  of 
decision  aids  and  decision  making  was  held  at  the 
interviewee's  option.  The  interview  was  designed  to  allow 
several  TACCO's  to  be  interviewed  simultaneously  but 
conversation  between  TACCOs  was  prevented.  The  interviews 
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were  designed  to  be 

completed 

in  a  two  hour  period 

,  as 

to 

minimize 

sample  bias  due  to 

interviewee  fatigue 

and 

to 

minimize 

work  time 

loss.  The 

designed  schedule 

was 

as 

follows : 

-  30  minutes  were  allowed  for  initial  briefing  and 
answering  questions 

-  20  minutes  were  allocated  for  the  psychometric 
sorting  task 

-  40  minutes  were  allocated  for  the  psychometric 
ranking  task 

-  30  minutes  were  used  to  discuss  the  interview  and 
suggested  improvements  in  the  area  of  decision 
aids. 

It  should  be  noted  that  the  TACCO's  interviewed  demonstrated 
an  active  interest  in  providing  fleet  inputs  to  an  area  many 
believed  to  be  critical  to  the  successful  accomplishment  of 
the  S-3A  ASW  mission. 

At  the  start  of  each  interview,  an  initial  briefing  and 
explanation  was  given,  outlining  air  ASW  history,  purpose  of 
the  interview  and  the  possible  impact  of  decision 
prioritization  on  the  implementation  of  the  future  decision 
aids  in  the  S-3A.  Questions  were  then  answered  and  the 
interviewees  were  introduced  to  the  two  tasks  they  were  to 
perform.  Portions  of  the  briefing  are  found  in  Appendix  A. 

The  actual  data  collection  effort  for  the  sorting  and 
ranking  tasks  were  performed  as  follows.  After  the  briefing. 
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each  TACCO  was  given  a  pack  of  cards  (similiar  to  Figure  1) 
and  a  set  of  response  forms  (reproduced  in  Appendix  B) .  The 
cover  sheet  of  the  response  form  requested  background 
information  on  each  TACCO  but  the  name  was  purposely  left 
off.  The  absence  of  names  from  the  response  forms  preserved 
anoniraity,  hopefully  leading  to  less  biased  responses.  The 
background  information  allowed  the  interview  to  insure  the 
requirement  of  experience  and  training  were  met  prior  to 
inclusion  in  the  test  sample.  Each  task  had  an  instruction 
sheet  preceeding  each  response  form,  explaining  the  required 
task  once  again.  The  unconstrained  sort  used  perceived 
similiarities  among  stimuli  as  the  criterion  for  group 
partitioning.  The  four  ranking  task  used  the  criterion 
discussed  previously.  Individual  questions  during  the  task 
were  discouraged  to  decrease  any  answer  interpretation  bias. 
All  subjects  finished  the  sorting  task  in  less  than  15 
minutes,  while  the  rankings,  normally,  required  10  minutes 
per  task. 

Following  the  last  ranking  task,  each  TACCO  had  an 
opportunity  to  comment,  regarding  ASW  decision  aids  in 
general,  and  S-3A  decision  aids  in  particular.  Comments  were 
solicited  concerning  the  interview  (new  ideas,  format, 
procedure)  and  are  discussed  in  the  Result  and  Conclusion 
sections  of  this  thesis. 
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IV.  DATA  ANALYSIS 


A.  OVERVIEW 

/ 

Chapter  Four  describes  the  data  analysis  for  this 
effort.  Initially,  the  sorting  data  was  preprocessed  using 
Burton's  [1975]  algorithm  into  a  dissimiliar i ity  matrix  to 
be  used  as  input  into  the  MDS  program.  The  ranking  data  was 
analyzed  through  the  use  of  correlation  techniques  and 
average  rankings  were  found.  The  dissimiliarity  matrix  was 
then  input  into  the  MDS  program  and  a  dimensional 
representation  of  the  decision  space  was  produced. 
Dimensional  interpretations  were  derived.  Next,  Unfolding 
Analysis  was  used  to  provide  a  representative  model  of  the 
ranking  decision  space.  Finally,  a  priority  scale  of  the 
decision  space  was  established  using  the  model  coefficients 
of  the  Unfolding  Analysis  and  the  Multidimensional  Scaling 
coordinates.  Figure  2  provides  an  overview  of  this 
methodology. 

B.  PRELIMINARY  ANALYSIS 

1.  Preprocessing  of  Sorting  Data 

Sorting  data  were  obtained  from  32  experienced  S-3A 
TACCOs,  however,  two  TACCOs  were  excluded  because  they  did 
not  have  the  experience  level  required  for  the  analysis.  The 
remaining  30  TACCOs  were  determined  to  be  'experienced'  for 
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GROUPS  DECISIONS  f  Ol  \  PROVIDES  NUMERICAL 


Figure  2.  Multidimensional  Scaling  prioritization 
Procedure  (adapted  from  Zachary  1980b) 


the  purposes  of  this  analysis.  These  30  sortings  are  listed 
in  Appendix  C.  The  30  sets  of  data  were  prepared  for  input 
into  the  MDS  computer  program  by  METRIC,  an  algorithm 
developed  by  Burton  (1975)  .  The  algorithm  processes  the 
unconstrained  sorting  data  to  produce  a  d i ss imi 1 i ar i ty 
measure  among  the  decision  functions  in  the  stimulus  set 
[Zachary  1980b] .  In  general,  a  dissimiliarity  measure  is  a 
binary  mapping  from  the  set  of  objects  scaled  to  some 
numerical  scale.  For  any  two  objects  i  and  j,  the 
dissimiliarity  between  i  and  j  is  expressed  as  d^j. 
Dissimiliarity  measures  has  three  main  properties: 


1) 

dij 

=  0  iff: 

i  =  j  : 

positivity 

dii 

>  0  otherwise 

2) 

dij 

*  dij  for 

all 

i,  j  :  symmetry 

3) 

dij 

i  dij  + 

d:x 

for  all  i,  j, 

triangle 


inequality. 

Dissimiliarity  measures  are  normally  represented  as 
a  square  matrix  where  each  row/column  represents  one  of  the 
stimuli.  Consequently,  the  terms  dissimiliarity  'measure' 
and  dissimiliarity  'matrix'  are  use  throughout  the  analysis 
interchangeably. 

Burton's  algorithm  computes  dissimiliarity  between 
two  stimuli  based  on  two  inputs: 

1)  the  number  of  subjects  who  place  them  in 
different  groups,  and 
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2)  the  values  assigned  to  the  parameters  a 
and  ( 

The  first  parameter,  a  ,  represents  the  level  of 
discrimination  selected  for  this  analysis  (as  discussed  in 
Chapter  III).  The  parameter,  (  ,  represents  the  minimal 

dissimiliarity  between  two  stimuli  that  results  when  two 
stimuli  are  placed  in  the  same  group  by  one  subject.  The 
algorithm  is  based  on  the  principle  that  each  subject  infers 
a  partition  on  the  set  of  stimuli.  R.  is  defined  as  the 
number  of  groups  formed  by  subject  i  and  is  defined  as 
the  number  of  stimuli  placed  in  group  k  by  subject  i.  If  the 
number  of  subjects  is  T,  then  dissimiliarity  between  stimuli 
x  and  y  is: 

’  2>  -  si  <*>  *>> 


where: 


V*'  y) 


’A^  if  subject  i  places  stimuli  x  and 
y  in  cell  k  of  the  partition, 

if  subject  i  places  stimuli  x  and  y 
in  different  cells  of  the  partition, 
and 

C  if  x  =  y. 


d(x,  y)  is  a  measure  of  dissimi 1 lari ty  which,  in  addition, 
is  a  metric  [Burton  1975].  There  are  two  constraints  on  d: 

1)  Bi  <  min  (A^k)  over  all  k 
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2)  C  >  max  (Aj^)  over  all  i,  k 
Additionally,  a  useful  measure  is  obtained  by  defining: 


Aik  - 

<Mik> 

Bik  = 

0 

c 

1  +  ( 

for 

a 

* 

0 

N  +  ( 

for 

a 

> 

0 

2  +  ( 

for 

a 

< 

0 

Since 

epsilon  ( 

t 

) 

is  a  minimum  value  used  to 

differentiate  items  placed  together  by  a  given  subject,  it 
should  be  very  small.  Alpha  (  a  )  represents  the  level  of 
distinction  used  by  the  TACCO  doing  the  sorting.  If  alpha  is 
greater  than  zero  (  o  >  0)  ,  a  greater  weight  is  given  to 
sortings  that  use  a  few,  large  cells,  and  the  distance 
measure  tends  to  be  the  lower  dimensionality  when  subjected 
to  Multidimensional  Scaling.  When  alpha  is  less  than  zero 
(  a  <  0)  #  a  greater  weight  is  given  to  partitions  which 
include  a  greater  number  of  small  cells,  and  the  measure 
results  in  higher  dimensional  MDS  solutions.  When  alpha 
equals  zero  (  a  *  0)  ,  equal  weight  is  given  large  and  small 
cell  partitions,  as  1  is  always  added  to  d(x,y)  whenever  x 
and  y  are  in  different  cells,  and  epsilon  is  added  to  d(x,y) 
when  they  are  place  in  the  same  cell. 

The  program,  METRIC,  performs  this  algorithm  to 
produce  a  d iss imi 1 iar i ty  matrix  from  the  unconstrained 
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sorting.  Because  the  intent  of  this  thesis  is  to  identify 
the  most  important  dimensions  vice  all  dimensions,  an  alpha 
value  of  -.5  was  used.  The  dissimiliar ity  matrix  is  shown 
in  Table  III. 

2.  Preliminary  Analysis  of  Rankings 

The  rankings  of  the  14  decision  functions  by  the 
four  criteria  discussed  in  Chapter  3  are  given  in  Appendix  D 
for  all  30  TACCOs.  The  requirement  of  the  preprocessing  of 
data,  necessary  in  the  sorting  data,  was  not  considered 
necessary  for  input  into  the  Unfolding  Analysis  procedure. 
However,  two  preliminary  analyses  of  the  ranking  data  were 
considered  useful  to  the  overall  analysis. 

The  first  preliminary  analysis  tested  the 
statistical  significance  of  the  ranking  data.  The  Kendall 
coefficient  of  concordance  :  W  is  a  statistic  which  measures 
the  relation  {degree  of  agreement)  among  several  rankings 
of  the  same  set  of  stimuli  [Seigal  1956].  The  coefficient  of 
concordance  is  an  index  of  the  divergence  of  the  actual 
agreement  shown  in  the  data  from  the  maximum  possible 
(perfect)  agreement.  A  test  of  significance  of  the 
coefficient  can  be  based  on  the  value  of  W  [Kendall,  1962]. 
This  test  is  based  on  the  fact  that  K (N  -  1)W  is  distributed 
according  to  the  Chi-Square  distribution  with  N  -  1  degrees 
of  freedom  (where  K  is  the  number  of  rankers  and  N  is  the 
number  of  stimuli).  The  null  hypothesis,  Hq  ,  and  the 
alternate  hypothesis  used  were: 
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Hq  :  there  is  no  agreement  among  the  subject's 
rankings  (i.e.  W  *  0)  . 

:  there  is  significant  agreement  among  the 
subject's  rankings  (i.e.  W  >  0)  . 

Table  IV  lists  the  coefficient  of  concordance  :  W, 

2 

the  X  statistic,  and  the  level  of  significance  for  each  of 
the  four  rankings.  All  four  rankings  were  found  to  be 
significant  at  less  than  the  .01  level,  rejecting  the  null 
hypothesis  with  greater  than  99  percent  confidence.  The 
significance  testing  indicates  that: 

1)  the  TACCOs  used  the  same  criterion  for  ranking, 
and 

2)  the  response  were  net  of  a  random  nature. 

The  second  preliminary  analysis  of  the  rankings 
provided  insight  into  the  interpretation  of  each  criterion 
(i.e.  1)  importance  to  mission  with  attack,  2)  importance  *-o 

mission  without  attack,  3)urgency,  and  4)workload).  Average 
rankings  were  used  as  a  tool  for  this  analysis.  Siegal 
[1956]  and  Kendall  [1962]  suggest  that  the  oest  estimate  of 
the  'true'  ranking  of  N  objects  is  provided,  v  :en  W  is 
significant,  by  the  order  of  the  various  sut. . of  the  -anks. 
An  extention  of  this  estimate  is  the  average  ranking. 
Average  rankings  yield  more  information  about  elative 
ranking  among  stimuli  than  does  the  sum  of  ranks  [Zachary 
1980b] .  The  average  rankings  are  shown  in  Tables  V  through 
VIII.  The  tables  show  a  significantly  different  ranking  for 
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each  criterion,  verifying  the  assumption  that  more  than  one 


criterion  was  necessary  to  capture  the  implicit 
prioritization  of  the  decision  functions  by  the  TACCOs. 


Table  IV. 

Significance 

of  Subject 

Agreement  in  Ranking 

RANKING 

CRITERION 

KENDALL 

COEFFICIENT 

CHI-SQUARE 

STATISTIC 

SIGNIFICANCE  LEVEL 
(FOR  REJECTION  OF  HQ) 

IMPORTANCE 

WITH 

ATTACK 

.382 

148.79 

<  .001 

IMPORTANCE 

WITHOUT 

ATTACK 

.468 

182.56 

<  .001 

URGENCY 

.558 

217.62 

<  .001 

WORKLOAD 

.243 

94.80 

<  .001 
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Table  V.  Average  Ranking  of  Decision  in  Mission 

With  Attack 


Gain  Attack  Criteria  3.866 

Classify  Signal  4.700 

Determine  Tarrget  Fix  5.400 

Determine  Aircraft  Weapon  Launch 
Position  5.433 

Determine  Weapons  Setting  For 

Attack  5.800 

Anticipate  Target  Movement  6.200 

Determine  Signal  Is  Valid  Contact  6.566 

Create  Sensor  Pattern  6.600 

Compensate  For  Propagation  Conditions  8.500 
Construct  Sensor  Monitoring  Pattern  8.665 

Extend  Sensor  Pattern  10.033 

Manage  Equipment  and  Stores  10.200 

Adjust  Pattern  to  Sensor  Failure  10.566 

Coordinate  Hand-Off  12.533 


Table  VI.  Average  Ranking  of  Decision  in  Mission 

Without  Attack 


Classify  Signal  3.900 

Anticipate  Target  Movement  4.966 

Create  Sensor  Pattern  5.066 

Determine  Signal  Is  Valid  Contact  5.166 

Determine  Target  Fix  5.266 

Coordinate  Hand-Off  5.666 

Compensate  For  Propagation  Conditions  6.500 
Construct  Sensor  Monitoring  Pattern  7.233 

Manage  Equipment  and  Stores  7.766 

Extend  Sensor  Pattern  8.366 

Adjust  Pattern  to  Sensor  Failure  8.733 

Gain  Atack  Criteria  10.600 

Determine  Aircraft  Weapon  Launch 
Position  13.033 

Determine  Weapon  and  Setting 

For  Attack  13.066 
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Table  VII.  Average  Ranking  of  Decision  by  Urgency 


Determine  Aircraft  Weapon  Launch 
Position  2.666 

Gain  Attack  Criteria  2.933 

Determine  Weapon  and  Setting 

For  Attack  4.800 

Anticipate  Target  Movement  5.266 

Determine  Target  Fix  5.333 

Determine  Signal  Is  Valid  Contact  5.766 

Classify  Signal  6.866 

Adjust  Pattern  to  Sensor  Failure  8.400 

Manage  Equipment  and  Stores  9.366 

Compensate  For  Propagation  Conditions  9.900 
Extend  Sensor  Pattern  10.266 

Create  Sensor  Pattern  10.666 

Coordinate  Hand-Off  11.366 

Construct  Sensor  Monitoring  Pattern  11.933 


Table  VIII.  Average  Ranking  of  Decision  by  Workload 


Gain  Attack  Criteria  4.266 

Determine  Target  Fix  4.733 

Anticipate  Target  Movement  5.333 

Determine  Aircraft  Weapon  Launch 
Position  5.566 

Determine  Weapon  and  Setting 

For  Attack  6.500 

Coordinate  Hand-Off  7.000 

Create  Sensor  Pattern  7.300 

Manage  Equipment  and  Stores  7.600 

Extend  Sensor  Pattern  8.166 

Adjust  Pattern  to  Sensor  Failure  8.633 

Compensate  For  Propagation  Conditions  9.100 
Construct  Sensor  Monitoring  Pattern  10.166 

Determine  Signal  Is  Valid  Contact  10.333 

Classify  Signal  10.333 
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C.  APPLICATION  OF  MDS  ALGORITHM 

The  dissimiliarity  matrix  discussed  in  the  preliminary 
analysis  was  the  primary  input  to  a  MDS  program.  The 
analysis  was  conducted  on  the  University  of  Pennsylvania 
Wharton  School's  DEC-10  computer  using  the  MDSX  package  of 
Multidimensional  Scaling  programs  [Coxon  et  al  1977],  This 
package  contains  several  different  algorithms  including  the 
MINASSA  algorithm  [Lingoes  and  Roskam  1975],  the  INDSCAL 
algorithm  [Carrol  and  Chang  1970]  and  the  TORSCA  IV 
algorithm  [Young  1968].  Zachary  [1980b]  used  each  of  these 
algorithms  on  similar  ASW  data  to  test  for  convergent 
validity  among  the  programs,  resulting  in  virtually 
identical  solutions.  Therefore,  in  the  dimensional  and 
prioritization  analysis,  the  MINASSA  algorithm  was  used 
because  the  other  two  algorithms  are  quasi-metric,  whereas 
MINASSA  is  totally  nonmetric,  that  is,  useful  in  analysing 
ordinal  data. 

The  MINASSA  algorithm  was  used  to  construct  solutions 
from  one  to  five  dimensions.  The  plot  of  the  goodness  of  fit 
measure  'stress'  versus  dimension  number,  shown  in 
Figure  3,  indicates  that  a  three  dimensional  solution  is 
best.  Kruskal  [1978]  suggest  that  if  the  number  of  stimuli 
considered,  N,  is  greater  than  four  times  the  dimension 
number,  D,  then  a  stress  measurement  of  .02  or  less  is 
considered  a  very  good  fit.  Additionally,  Roskam's  [1977] 
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Figure  3.  MDS  Stress  vs.  Dimensionality 
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'rule-of-thumb'  for  assessing  the  stress  of  a  MINASSA 
solution  indicates  that  a  stress  between  .05  and  .01  is 
considered  very  good  quality.  Therefore,  with  a  stress  of 
.018,  the  three  dimensional  solution  satifies  the  stress 
requirement  for  a  good  fitting  model.  Although  the  4th  and 
5th  dimensional  solutions  also  provide  stress  measurements 
less  than  .02,  the  incresed  complexity  of  the  solutions  due 
to  higher  dimensionality  did  not  merit  further 
consideration . 

The  three  dimensional  solution  is  shown  in  Figure  4.  For 
ease  of  interpretation,  two  dimensional  plots  of  each 
possible  pair  of  dimensions  are  given  in  Figures  5  through 
7.  The  plots  represent  relative  locations  in  two  dimensions 
of  each  of  the  14  decision  functions.  Table  IX  shows  the 
actual  coordinates  of  each  decision  function  in  three 
dimensions . 

D.  DIMENSIONAL  INTERPRETATION 

The  MINASSA  algorithm  reconstructs  the  underlying 
features  of  relationships  of  the  decision  functions  as  the 
dimensional  axes  of  the  MDS  solution.  Unfortunately,  the 
algorithm  cannot  identify  the  meaning  of  these  dimensional 
axes.  The  meanings  of  the  axes  must  be  interpreted  through 
careful  examination  of  the  axes  individually,  through 
unidimensional  projection  of  each  coordinate  set,  and  the 
interaction  of  the  axes.  The  unidimensional  projections  are 
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shown  in  Figures  8  through  10. 


The  multidimensional  representation  constructed  by 
MINASSA  is  unique  and  translation  or  rotation  of  the  axes 
used  to  obtain  a  more  interpretable  set  of  projections  did 
not  change  the  relationship  among  sitmuli.  However,  the 
interpretations  of  the  dimensions,  discuss  subsequently,  are 
based  on  the  'unrotated  -  untranslated'  spatial 
representations  given  in  Figures  5  through  7.  The  basis  for 
this  representation  is  two-fold.  First,  the  MINASSA  solution 
is  based  on  an  implicit  rotation  performed  by  the  algorithm 
to  minimize  dependence  among  dimensions.  The  orientation  of 
the  dimension  axes  provided  the  most  independent 
representation  in  terms  of  the  set  of  decision  functions. 
Consequently,  the  independent  nature  of  the  resulting  axes 
enhanced  the  interpretability  of  the  dimensions.  Secondly,  a 
rotation  and  translation  of  the  axis  was  attempted  to 
provide  a  more  interpretable  representation.  This  analysis 
did  not  add  to  the  interpretability  of  the  spatial 
representation.  Additionally,  because  the  unrotated 
orientations  is  based  on  the  criterion  of  minimizing  the 
interdimensional  correlation,  the  original  MINASSA  result  is 
used  in  the  following  interpretation. 
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Figure  4.  Three  Dimensional  MDS  Plot 
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Figure  5.  Dimension  1  vs  Dimension  2 
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Figure  6.  Dimension  1  vs  Dimension  3 
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Figure  7.  Dimension  2  vs  Dimension  3 
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Table  IX.  Final  Multidimensional 

Scaling  Coordinates 

Dimension 

Dimension 

Dimension 

1 

2 

3 

Adjust  Pattern  to 

Sensor  Failure 

.0447 

-.5791 

.1632 

Extend  Sensor  Pattern 

.1172 

-.5526 

-.5211 

Anticpate  Target 
Movement 

2909 

.7820 

-.2064 

Construct  Sensor 
Monitoring  Pattern 

.1720 

-.6017 

-.2467 

Coordinate  Hand-Of£ 

1.4256 

-.4035 

.6366 

Determine  Signal  Is 
Valid  Contact 

-1.3964 

.0029 

.2927 

Determine  Weapon 

Setting  For  Atack 

.6826 

.6964 

.0826 

Gain  Attack  Criteria 

.4285 

.7912 

-.0787 

Determine  Target  Fix 

-.5060 

.7070 

-.1407 

Create  Sensor  pattern 

.2664 

-.5176 

-.5525 

Manage  Equipment 
and  Stores 

.0169 

-.4266 

.6182 

Classify  Signal 

-1.3337 

-.0282 

.3539 

Compensation  for 
Propagation  Conditions 

-.2605 

-.6237 

-. 3427 

Determine  Aircraft 
Launch  Position 

.6336 

.7694 

-.0585 

55 


10 


Z 

o 

(A 

Z 

ui 

| 

5 


i  i  i  i  i  i  i  i  i  i  i  i  i  i 


56 


Figure  8.  Dimension  1  Unidimensional  Plot 
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Figure  9.  Dimension  2  Unidimensional  Plot 
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Figure  10.  Dimension  3  Unidimensional  plot 
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Interpretation  of  Dimension  One;  Workload 
This  dimension  was  interpreted  as  representing  the 
workload  of  the  TACCO  (physical  and  cognitive)  when 
executing  the  decision  function  (Figure  8).  The  extreme 
points  of  this  mapping  -  Coordinate  Hand-off,  Classify 
Signal,  and  Determine  Signal  is  Valid  Contact  lend 
reinforcement  to  this  interpretation.  Although  the  TACCO  is 
ultimately  responsible  for  the  successful  completion  of  the 
tactical  mission,  the  sensor  operators  (acoustic  and 
nonacoustic)  are  required  to  Classify  Signal  and  Determine 
Signal  is  Valid  Contact.  The  TACCO  may  verify  the  sensor 
operator's  decision  but  as  far  as  workload  of  those  decision 
functions  to  the  TACCO  is  concerned,  the  workload  is 
minimal.  The  workload  of  these  decision  functions  lie  in  the 
interpretation  and  analysis  of  target  signals,  which  is  more 
of  a  sensor  operator's  function.  On  the  other  extreme  is 
perhaps  the  heaviest  workload  decision  function  of  the 
decision  set  -  Coordinate  Hand-off.  Many  inputs  are  required 
to  be  processed  and  assimiliated  to  provide  a  well  executed 
Coordinated  Hand-off.  Although  the  format  of  a  Coordinated 
Hand-off  is  well  established,  the  degree  of  detail  required 
to  maintain  target  contact  is  great. 

The  implicit  ordering  of  decision  functions  between 
these  extreme  stimuli  support  the  workload  interpretation  of 
this  dimension.  The  next  three  highest  decision  functions 
after  Coordinated  Hand-off  -  Determine  Weapon  and  Setting, 
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Determine  Aircraft  Weapon  Launch  Position,  and  Gain  Attack 
Criteria  -  involve  great  complexity.  Because  the  actual 
deptn  of  the  target  is  difficult  to  assess,  the  TACCO  must 
use  every  input  to  the  problem  to  determine  the  correct 
weapon  selection  and  its  search  setting  in  order  to  maximize 
probability  of  kill.  Maneuvering  the  aircraft  into  weapons 
launch  position  and  gaining  attack  criteria  must  occur 
within  a  small  window  in  time  in  order  to  place  a  weapon  on 
target.  A  small  miscalculation  in  the  execution  of  the 
decision  functions  could  cause  the  aircraft  to  be  out  of 
position  when  attack  criteria  is  gained  resulting  in  a  lost 
attack  opportunity. 

A  cluster  of  five  decision  functions  concerning 
sensor  placement,  are  found  close  to  the  zero  point  -  Create 
Sensor  Pattern,  Construct  Sensor  Monitoring  Pattern,  Extend 
Existing  Sensor  Pattern,  Adjust  Pattern  to  Sensor  Failure, 
and  Manage  Equipment/Stores  to  Accomodate  Present  and  Future 
Needs.  All  of  these  decision  functions  exhibit  two  levels  of 
cognitive  workload.  The  more  cognitive  level  concerns  the 
situational  inputs  of  each  function  -  the  decisions  that 
change  with  time,  both  present  and  future.  The  less 
cognitive  level  represents  the  computer  controlled  decision 
inputs  such  as  monitoring  sequence,  search  pattern,  and 
stores  remaining,  and  the  formatted  procedures  established 
by  the  On-Scene  Commander  (such  as  buoy  spacing,  type  of 
pattern,  and  buoy  channel  numbers) . 
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The  three  remaining  decision  functions  -  Compensate 
for  Acoustic/Atmospheric  Propagation  Conditions,  Anticipate 
Target  Movement,  and  Determine  Target  Fix  require  less  work 
with  respect  to  the  TACCO  than  the  'zero'  cluster  but  more 
work  than  the  extreme  signal  processing  functions.  The 
computer  on  board  the  S-3A  has  a  bathythermal  and  a  radar 
range  program  that  takes  external  inputs  to  establish 
situational  conditions.  Anticipating  Target  Movement  can  be 
assisted  by  the  prediciton  and  ranging  computer  functions 
and  fixing  information  is  supplied  by  the  sensor  operators 
with  confidence  levels. 

2.  Interpretation  of  Dimension  Two;  Time  Criticality 

Dimension  two  was  interpreted  as  representing  the 
degree  of  time  criticality  of  the  decision  functions.  The 
unid? mensional  representation  (see  Figure  9)  shows  three 
well  defined  groupings  of  decision  functions,  helpful  in 
analyzing  this  dimension.  The  most  neutral  decision 
functions  on  the  criticality  dimension  are  Deterimine  Signal 
is  a  Valid  Contact,  amd  Classify  Signal.  These  signal 
processing  functions  are  considered  somewhat  time  critical 
in  determining  the  tactics  to  use  against  the  target  but 
less  critical  concerning  the  prosecution  of  the  target. 

On  the  high  criticality  extreme  of  the 
representation,  the  actual  tactical  decision  functions  - 
Gain  Attack  Criteria,  Anticipated  Targert  Movement, 
Determine  Aircraft  Weapons  Launch  Position,  Determine  Target 
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Fix,  and  Determine  Weapon  and  Setting  for  Attack  -  are 
closely  clustered.  The  relative  differences  between  the 
decision  functions  were  very  small  numerically,  suggesting 
little  difference  in  the  degree  of  criticality.  However,  the 
dynamic  nature  of  these  decision  functions  requires 
expeditious  scrutiny  of  all  time  critical  inputs  to  reach 
the  most  optimal  decision. 

The  remaining  decision  functions  were  clustered  at 
the  low  criticality  extreme  of  the  representation.  This 
stimuli  subset  -  Coordinate  Hand-off,  Manage 
Equipment/Stores  to  Accomodate  Present  and  Future  Needs, 
Create  Sensor  Pattern,  Extend  Sensor  Pattern,  Adjust  Sensor 
Pattern  to  Sensor  Failure,  Construct  Sensor  Monitoring 
Pattern,  and  Compensate  for  Acoustic/Atmospheric  Propagation 
Conditions  -  is  very  important  in  the  accomplishment  of  the 
ASW  mission.  However,  time  criticality  is  not  normally  a 
factor  in  making  any  of  these  decisions. 

Speed  of  decision  making  can  be  considered 
synomymous  with  time  criticality  for  this  dimensional 
interpretation.  Figure  9  illustrates  areas  requiring  fast, 
moderate,  and  more  deliberate  decision  making. 

3 .  Interpretation  of  Dimension  Three:  Complexity 

Dimension  three  was  interpreted  as  referring  to  the 
degree  of  complexity  in  the  decision  function  with  respect 
to  the  TACCO.  Figure  10  shows  a  rather  scattered 
unidimensional  plot  of  the  decision  function.  Two  small 
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clusters  on 


either  extreme  and  an  evenly  distributed 
grouping  of  the  remaining  decision  functions  about  zero  made 
this  dimension  the  most  difficult  to  interpret. 

The  decision  functions  on  the  high  extreme  of 
Complexity  -  Coordinate  Hand-off,  and  Manage 
Equipment/Stores  to  Accomodate  Present  and  Future  Needs 
showed  high  degrees  of  decision  complexity,  assembling 
information  for  hand-off  and  inventorying  sensors.  While  on 
the  low  extreme  of  the  complexity  dimension,  the  stimuli 
Extend  Sensor  Pattern  and  Create  Sensor  Pattern  -  required 
little  original  thought  and  simply  followed  standard  tactics 
procedures.  The  complexity  of  either  decision  function  was 

minimal . 

The  remaining  decision  functions  are  evenly  spaced 
around  zero  suggesting  moderate  complexity.  The  stimuli 
Classify  Signal,  and  Determine  Signal  is  a  Valid  Contact, 
has  a  greater  degree  of  complexity,  processing  all  inputs  to 
those  decision  functions.  On  the  other  end  of  the  grouping. 
Compensate  for  Acoustic/Atmospheric  Propagation  Conditions, 
minimizes  complexity  through  computer-assistance  programs. 
The  rest  of  the  'zero’  grouping  involve  implicit  degrees  of 
complexity. 
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E.  PRIORITIZATION  OF  THE  DECISION  SPACE 
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The  three  dimensions  constructed  by  the  MDS  analysis  of 
the  sorting  data  represent  a  geometric  model  of  experienced 
S-3A  TACCOs'  understanding  of  Air  ASW  decisions.  These  three 
dimensions  represent  the  basic  implicit  relationships  that 
TACCOs  used  in  the  Air  ASW  decision  making.  Therefore,  other 
relationships  or  characteristics  should  exist  only  as 
combinations  of  these  underlying  features.  One  such 
characteristic  is  priority  or  importance.  Two  of  the  rank 
orderings,  discussed  earlier,  had  as  criterion  -  importance 
to  the  mission.  This  section  discusses  the  implicit 
combination  of  these  three  dimensions  used  by  the  TACCOs 
when  they  ranked  the  14  decision  functions  using  the 
importance  criterion  (both  with  attack  and  without  attack). 

The  basic  technique  used  to  determine  TACCO  priority 
functions  is  termed  Priority  Mapping,  and  has  been  developed 
directly  for  this  type  of  application  by  Zachary  [1980b],  It 
is  related  to  the  psychometric  techniques  known  as 
Preference  Mapping  [Carrol  and  Chang  1967]  and  is  based  on 
Coombs  [1950]  and  Bennett  and  Hayes  [1960]  notion  of 
Unfolding  Analysis. 

1.  Unfolding  Analysis  and  Priority  Mapping 

In  the  multidimensional  model  of  the  Air  ASW 
decision  space,  each  decision  function  in  the  stimulus  set 
has  a  unique  projection  or  coordinate  on  each  of  the 
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dimensional  axes.  Each  dimension  is  considered  to  be  a  scale 
which  measures  some  relevent  perceptual  attribute  of  the 
decision  space.  Collectively,  the  three  scales  in  this  model 
measure  all  of  the  independent  features  needed  to  represent 
the  perceived  interrelationships  among  decision  functions  in 
the  decision  space.  In  this  model,  the  three  dimensions 
provide  numerical  measures  of  all  the  important  features  on 
which  experienced  S-3A  TACCOs  perceive  ASW  decisions  (as 
shown  in  Table  IX)  . 

Therefore,  other  salient  characteristics  of  the 
decision  functions  must  be  combinations  of  the  features 
represented  by  the  axes  of  the  MDS  model.  These  derivative 
characteristics  or  features  may  have  differing  measurement 
scales,  resulting  in  different  representations  within  the 
multidimensional  space.  The  measurement  scales  include 
categorical,  ordinal,  and  interval-ratio.  The  categorical 
scale  partitions  the  multidimensional  space  into  unordered 
regions.  The  ordinal  scale  provides  a  graded  or  ordered 
partition  of  the  multidimensional  space.  Finally,  the 
interval-ratio  scale  represents  the  feature  as  a  directed 
vector  or  curve  passing  through  the  multidimensional  space. 

Unfortunately,  little  can  be  done  to  represent 
categorical  derivative  features  in  the  multidimensional 
model.  However,  through  the  use  of  Unfolding  Analysis,  it  is 
possible  to  construct  a  representation  of  ordinal  or 
interval-ratio  derivative  features.  The  technique  is  based 
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on  the  concept  that  the  general  form  of  the  representation 
of  a  derivative  feature  in  a  predefined  multidimensional 
space  can  be  determined  by  comparing  different  models  of 
increasing  complexity  [Zachary  1980b] . 

Unfolding  Analysis  restricts  itself  to  four  specific 
formulae  which  have  simple  mathematical  forms  and  direct 
psychological  interpretations.  Although  the  analysis  could 
be  performed  on  either  ordinal  or  interval-ratio  scale 
features,  the  analysis  used  in  this  discussion  was  limited 
to  ordinal  scales,  consistent  with  the  (ordinal)  ranking 
data . 

In  Unfolding  Analysis,  the  four  models  are 
considered  and  compared  in  terms  of  their  ability  to 
represent  a  given  ordinal  scale  in  a  multidimensional  space. 
Usually,  the  ordinal  scale  is  a  ranking  of  a  stimulus  set  by 
individuals  according  to  some  criterion  and  the 
multidimensional  space  is  a  result  of  some  MDS  analysis. 
Unfolding  Analysis  attempts  to  represent  the  ranking 
criteria  as  a  function  of  the  MDS  dimensionalization. 

Techniques,  similar  to  regression,  can  be  used  to 
define  an  equation  which  models  the  criterion  representation 
as  a  function  of  the  MDS  dimensions.  Carroll  and  Chang 
[1977]  developed  a  set  of  computational  procedures  for 
performing  these  regression-like  analysis.  In  this  analysis, 
their  computational  approach  is  used  to  model  data  on 
perceived  priority  of  decision  functions.  This  adaptation  of 
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the  model  is  appropriately  termed  ‘Priority  Mapping',  since 
it  constructs  a  mathematical  'map'  of  the  TACCOs  intuitive 
prioritization  of  the  decision  functions. 


2.  Four  Models  of  Priority 

The  Unfolding  Analysis  and  Priority  Mapping 
procedures  employ  four  models  of  priority.  These  models  are 
based  on  a  heirarchial  structure  such  that  each  model  is  a 
special  case  of  the  next  'higher'  model.  Each  model  in  the 
heirarchy  subsumes  all  subordinate  models.  The  models  will 
be  explained  in  terms  of  ordinal  representations  consistent 
with  the  data  set. 

An  ordinal  scale  in  a  multidimensional  space  divides 
the  space  into  graded  regions,  such  that  a  point  in  one 
region  has  a  different  rank  than  a  point  in  another  region, 
but  the  same  rank  as  another  point  in  the  same  region.  In 
other  words,  the  space  is  broken  into  isopriority  regions 
separated  by  isopriority  contours.  A  geometric 
representation  of  the  four  models  showing  the  isopriority 
contour  structure  is  shown  in  Figure  11.  For  simplicity. 
Figure  11  and  the  following  discussion  are  expressed  in 
terms  of  a  two-dimensional  space.  A  generalization  of  this 
approach  can  be  extended  to  three  or  more  dimensions. 

The  first  model  (and  lowest  in  the  heirarchy)  is  the 
vector  model.  This  model  assumes  that  each  dimension 
contributes  in  a  linear  fashion  resulting  in  a 
prioritization  defined  by  stimuli  projections  onto  a  vector 
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WEIGHTED  DISTANCE  MODEL  GENERALIZED  DISTANCE  MODEL 


Figure  11.  Four  Models  of  Priority 
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in  the  multidimensional  space.  The  isopriority  contours  are 
lines  (in  this  case)  perpendicular  to  the  stimulus  vector 
(shown  in  Figure  11a) .  The  vector  model  is  similiar  to  the 
concept  of  Multiattribute  Utility  (MAU) .  Each  dimension 
(attribute)  contributes  to  the  model  in  a  linear  way  with 
the  relative  contribution  of  each  dimension  defined  by  a 
coefficient  assigned  to  it.  The  vector  model  can  be  compared 
to  the  economic's  'more  is  better*  model,  because  it  assumes 
that  whatever  the  dimension,  more  of  it  will  contribute  to  a 
higher  priority. 

The  remaining  three  models  define  priority  in  terms 
of  distance  in  the  multidimensional  space.  These  models  are 
based  on  the  assumption  that  there  is  an  ideal  point 
somewhere  in  the  modeled  space,  such  that  the  closer  a 
stimulus  is  to  the  ideal  point,  the  higher  the  priority  of 
that  stimulus.  The  simplest  of  the  distance  models  is  termed 
the  Unweighted  Distance  Model  or  just  Distance  model.  This 
model  prioritizes  stimuli  strictly  according  to  their 
distances  from  the  ideal  point.  This  distance  measurement 
for  this  model  is  Euclidean  in  nature  yeilding  isopriority 
contours  represented  by  concentric  circles  around  the  ideal 
point  (as  shown  in  Figure  lib) .  The  unweighted  model  assumes 
each  dimension  has  an  equal  contribution  to  priority. 

A  more  general  representation  in  terms  of  distance 
allows  the  distance  between  the  ideal  point  and  each 
decision  to  be  a  function  of  weighted  dimensionality.  More 
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specifically,  dimensions  may  not  contribute  equally  to  the 
model  so  each  dimensional  axes  is  given  a  relative  weight. 
This  model,  termed  the  Weighted  Distance  Model,  allows  the 
isopriority  contours  to  assume  the  general  slope  of  an 
ellipse  parrallel  to  the  dimensional  axes  as  shown  in  Figure 
11c. 

In  the  models  previously  discussed,  an  assumption  is 
that  each  dimension  contributes  independently  to  priority. 
However,  interaction  may  occur  among  some  or  all  dimensions 
in  constituting  the  concept  of  priority.  To  allow  such 
interactions  among  dimensions,  a  model  can  be  constructed  by 
permitting  the  axes  to  be  rotated  before  the  distances  are 
computed.  This  model,  termed  the  Generalized  Distance  Model, 
allows  each  dimension  to  contribute  independently  and 
collectively  to  the  priority  of  an  decision  function.  The 
isopriority  contours  are  similar  to  the  weighted  distance 
model  except  the  contours  can  assume  any  orientation  in  the 
space  as  shown  in  Figure  lid. 

These  four  models  -  the  vector  model,  the  distance 
model,  the  weighted  model,  and  the  generalized  model  -  are 
the  mathematical  basis  for  the  Unfolding  Analysis  and 
Priority  Mapping.  In  Unfolding  Analysis,  the  ability  of  each 
model  to  replicate  the  ordering  of  the  decisions  by  the 
TACCOs,  is  evaluated  and  the  most  applicable  model  is 
selected.  In  Priority  Mapping,  the  specific  coefficients  of 
the  selected  model  are  determined  and  a  precise  mathematical 
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prepresentation  of  priority  is  constructed. 


F.  UNFOLDING  ANALYSIS  AND  PRIORITY  MAPPING  OF  DECISION 
FUNCTIONS 

Unfolding  Analysis  procedures  were  conducted  for  the  two 
sets  of  ranking  of  decision  functions  by  importance.  The 
first  analysis  considered  the  rankings  by  importance  in  the 
ASW  mission  where  the  goal  was  an  attack  on  the  submarine. 
The  individual  rankings  are  in  Appendix  D,  and  the  average 
rankings  are  given  in  Table  V.  The  second  analysis 
considered  the  rankings  by  importance  to  the  ASW  mission 
where  the  goal  was  surveillance  of  the  submarine.  The 
average  rankings  for  this  criterion  is  given  in  Table  VI  and 
the  individual  rankings  are  in  Appendix  D.  The  priority 
mapping  was  conducted  with  the  PREFMAP  program  developed  by 
Carrol  and  Chang  [1967,  1977],  which  performs  the  Unfolding 
Analysis  as  described  in  the  last  section.  PREFMAP,  like 
MINASSA,  is  a  part  of  the  MDSX  package  of  Multidimensional 
Scaling  programs  [Coxon  et  al  1977]  available  on  the  Wharton 
School's  DEC-10  Computer.  PREFMAP  performs  Unfolding 
Analysis  by  conducting  seperate  analysis  of  each  individual 
ranking  against  the  multidimensional  configuration  of  the 
stimulus  set  provided  by  the  user. 

1.  Prioritization  of  Decision  Function  by  Importance  in 
Mission  with  Attack 

Using  the  PREFMAP  program,  the  analysis  began  by 
constructing  trial  representations  of  the  rankings  criterion 
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using  each  of  the  four  models  discussed  earlier.  These 

representations  produced  different  rankings  of  the  decisions 

functions,  which  can  be  compared  to  the  actual  TACCO 

rankings  to  determine  the  appropriateness  of  each  model.  The 

simplest,  most  reasonable  measure  of  comparison  is  the 

correlation  between  the  actual  rankings  and  the  rankings 

produced  by  the  model.  The  correlation,  when  squared, 

becomes  the  coefficient  of  variation,  r  ,  indicating  the 

proportion  of  variation  in  the  TACCO  rankings  accounted  for 

in  the  PREFMAP  rankings.  A  F  test  statistic  can  be  computed 
.  2 

from  r  and  used  to  test  the  null  hypothesis,  Hq,  that  the 
value  of  r  is  strictly  a  random  result.  The  values  of  r  , 
the  F  ratio,  and  the  levels  of  significance  are  given  in 
Table  X. 


Table  x.  Significance  of  Results  for  Mission  with  Attack 


MODEL 

~-r- 

F  STATISTIC 

REJECTION  LEVEL 

GENERALIZED 

.996 

116.8 

<.01 

WEIGHT  DIST. 

.982 

62.6 

<.01 

UNWEIGHTED 

.943 

37.4 

<.01 

DISTANCE 

VECTOR 

.939 

51.2 

<.01 

All  four  models  produced  a  significant  (non-chance) 
representation  of  the  TACCO  data.  The  next  phase  of  the 
analysis  is  to  determine  which  is  the  best  representation. 

The  most  widely  used  criterion  for  selecting  among 
models  is  the  goodness  of  fit  measurement  between  the  model 
and  the  data.  The  goodness  of  fit  is  measured  in  all  four 
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2  2 
models  by  r  -  the  coefficient  of  variation.  The  r  value 

for  all  four  models  is  greater  than  .99,  indicating  a  good 

fit  by  all  models.  In  fact,  each  model  in  the  heirarchy 

accounts  for  more  variation  than  the  lower  models.  This 

reinforcing  result  is  an  artifact  of  the  PREFMAP  algorithm 

since  each  heirarchial  model  increases  the  complexity  of  the 

equation  with  increases  in  the  number  of  parameters. 

Consequently,  the  model  with  more  parameters  will  explain 

more  of  the  variance  than  the  subordinate  models.  Therefore, 

it  is  reasonable  to  assume  that  r  will  increase  as  the 

heirarchy  is  traversed  from  the  vector  to  the  generalized 

model.  The  fundamental  question  is  whether  or  not  the 
2 

increase  in  r  ,  as  the  model  heirarchy  is  traversed,  is 
significant  as  the  number  of  model  parameters  increase. 

To  answer  this  question,  a  statistical  comparison 
among  the  model  rankings  was  used.  A  F  statistic  was 
computed  from  the  r  values  of  each  pair  of  models  (the 
between-model  coefficient  of  variation)  to  test  the  null 
hypothesis  that  the  increase  in  the  value  of  r  as  the  model 
heirarchy  is  traversed  is  only  a  chance  result.  A  failure  of 
this  hypothsis  for  a  given  pair  of  models  indicated  that 
there  was  not  a  significant  difference  in  the  explanatory 
power  of  the  two  models  and  the  subordinate  model  would  be 
preferred  because  it  is  less  complex.  The  comparison  among 
the  four  models  is  shown  in  Table  XI. 


i i 
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Table  XI.  Comparison  of  Coefficients  of  Variation  for 

Mission  with  Attack 


MODEL 


F  STAT 

SIGN. 


WEIGHTED 

5.1 

.022 

UNWEIGHTED 

11.2 

.018 

7.4 

.019 

VECTOR 

10.1 

5.5 

21.0 

.021 

.030 

.424 

GENERALIZED  WEIGHTED  UNWEIGHTED 

As  illustrated  in  Table  XI,  the  generalized 
distance  model  is  significantly  more  powerful  than  the  other 
models  with  a  confidence  of  greater  than  95  percent. 
Therefore,  the  generalized  distance  model  was  selected  to 
represent  the  TACCO  rankings  according  to  this  criterion. 

Having  selected  the  'best'  model  for  representing 
the  ranking  criterion,  a  precise  priority  function  was 
derived  using  the  PREFMAP  program.  The  importance  of 
priority  of  a  decision  function  j  in  a  mission  with  attack 
(pwa  (dj>)  was  given  by: 

Pwa  j5  -  -•148x2jl  -1.74x2j2  -.417x2j3 


+.330xjlXj2  +.774xjlxj3  -1.20xj2xj3 
+  .018Xj1  -. 091x j  2  -. 670Xj  3  +.727 


Xjl 

is 

the 

coordinate 

of  dj 

on 

dimension 

1 

X  j  2 

is 

the 

coordinate 

of  d j 

on 

dimension 

2  and 

X  j  3 

is 

the 

coordinate 

of  d j 

on 

dimension 

3. 

The 

values  of  Xj^, 

x  j  2 ,  x 

j  3 

are  given 

in  Table  IX. 
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The  priority  values  of  the  fourteen  decision  function 
generated  by  this  function  are  given  in  Table  XII. 


Table  XII.  Decision  Function  Prioritization  for 
Mission  with  Attack 


PRIORITY 

RANK 

NAME 

PRIORITY 

SCORE 

1 

GAIN  ATTACK  CRITERIA  (AC) 

-.2489 

2 

CLASSIFY  SIGNAL  (CS) 

-.2464 

3 

DETERMINE  SIGNAL  IS  VALID 

CONTACT  (VC) 

-. 1947 

4 

DETERMINE  AIRCRAFT  WEAPON 

LAUNCH  POSITION  (LP) 

-.1852 

5 

DETERMINE  WEAPON  AND  SETTING 

FOR  ATTACK  (DS) 

-.1785 

6 

ANTICIPATE  TARGET  MOVEMENT  (TM) 

-.1431 

7 

DETERMINE  TARGET  FIX  (TF) 

-.1198 

8 

CREATE  SENSOR  PATTERN  (CP) 

.0402 

9 

CREATE  SENSORING  MONITORING 

PATTERN  (MP) 

.0444 

10 

EXTEND  SENSOR  PATTERN  (EP) 

.0673 

11 

COMPENSATE  FOR  PROPAGNATION 

CONDITIONS  (PC) 

.1368 

12 

ADJUST  PATTERN  TO  SENSOR 

FAILURE  (AP) 

.1867 

13 

MANAGE  EQUIPMENT  AND  STORES  (ME) 

.1980 

14 

COORDINATE  HAND-OFF  (CH) 

.3656 

These  values  represent  the  relative  priority  of  the 
decision  functions  in  a  mission  with  attack.  The  lower 
values  (i.e.,  negative)  in  Table  XII  indicate  a  higher 
priority  and  the  higher  values  represent  a  lower  priority. 
This  ordering  is  the  result  of  the  TACCOs  assigning  lower 
ranks  to  the  more  important  decisions  and  higher  ranks  to 
the  less  important  decisions  (PREFMAP  program  merely 
maintains  this  directionality  in  its  computations). 
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2.  Prioritization  of  Decision  Function  by  Importance  in 
Mission  Without  Attack 

An  identical  analysis  was  conducted  for  the  TACCO 

rankings  of  decision  functions  where  the  criterion  was 

importance  to  ASW  mission  without  attack.  PREFMAP 

constructed  the  initial  representations  to  determine  which 

of  the  four  models  produced  'best'  representation  of  the 

2 

TACCO  data.  The  coefficient  of  variation  (r  )  and  the 
statistical  test  (F)  (Table  XIII)  were  used  to  determine 
which  model  is  the  best  representation. 


Table  XIII.  Significance  of  Result  for  Mission 
Without  Attack 


MODEL 

r2 

F  STATISTIC 

REJECTION  LEVEL 

GENERALIZED 

.997 

130.9 

<.01 

WEIGHT  DIST. 

.949 

21.8 

<.01 

UNWEIGHTED 

DISTANCE 

.881 

16.7 

<.01 

VECTOR 

.795 

13.0 

H  | 
O  1 
■  | 
V  I 

As  with  the  previous  criterion,  the  null  hypothesis 
can  be  rejected  for  all  four  models,  indicating  that  a 
significant  representation  of  the  TACCO  ranks  were  provided 
by  all  models. 

As  before,  the  four  models  were  compared  against 
each  other  to  determine  the  model  of  best  representation. 
The  'between  model'  F  statistic  v as  used,  testing  the  null 
hypothesis,  Hq ,  that  the  increase  in  the  proportion  of 
variance  accounted  for  by  the  more  powerful  model  was  a 
chance  result  accounted  for  by  its  increased  number  of 


parameters.  Table  XIV  show  the  ' between-model '  F  statistic 
and  the  associated  significance  level. 


Table  XIV. 

Comparison  of 
for  Mission 

Coefficients  of 
Without  Attack 

Variation 

nUULL 

F  STAT 

SIGN. 

WEIGHTED 

18.7 

.008 

UNWEIGHTED 

27.3 

.003 

4.7 

.051 

VECTOR 

39.6 

.002 

7.0 

.016 

6.5 

.031 

GENERALIZED  WEIGHTED  UNWEIGHTED 


There  were  more  pronounced  differences  in  the 
significance  levels  between  models  in  Table  XIV  than  in 
Table  XI.  More  specifically,  the  Generalized  Distance  Model 
was  found  to  be  significantly  better  in  representing  the 
TACCO  data  with  a  confidence  of  greater  than  99  percent.  As 
a  result,  the  Generalized  Distance  Model  was  selected  as  the 
most  appropriate  model  for  the  no  attack  criterion. 

As  in  the  previous  analysis,  the  PREFMAP  program  was 
used  to  'map1  the  precise  mathematical  priority  function. 
Using  this  algorithm,  the  importance  of  decision  j  on  a 
mission  without  attack  (pwoa(dj))  was  given  by: 

Pwoa(dj>  *  -230x2jl  -1.93*2j2  -.815x2j3 


+.623Xj1Xj2  +.385Xj1Xj3  -.592Xj2xj3 

+  .226Xj1  +. 260x j 2  -.410Xj3  +.888 


i 
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X. 

X. 


where  Xj^  is  the  coordinate  of  decision  j  on  dimension  1 

ij2  is  the  coordinate  of  decision  j  on  dimension  2  and 

is  the  coordinate  of  decision  j  on  dimension  3. 

The  values  of  the  Xj^,  as  before,  were  given  in 

Table  IX.  The  priority  values  for  the  fourteen  decision 
functions  are  given  in  Table  XV. 


Table  XV.  Decision  Function  Prioritization  for  Mission 

Without  Attack 


PRIORITY 

RANK 

NAME 

PRIORITY 

SCORE 

1 

CLASSIFY  SIGNAL  (CS) 

-.2309 

2 

DETERMINE  SIGNAL  IS  VALID 

CONTACT  (VC) 

-.2210 

3 

DETERMINE  TARGET  FIX  (TF) 

-. 1612 

4 

ANTICIPATE  TARGET  MOVEMENT 

-.1473 

5 

COORDINATE  HAND-OFF  (CH) 

-.1246 

6 

EXTEND  SENSOR  PATTERN  (EP) 

-.0637 

7 

CREATE  SENSOR  PATTERN  (CP) 

-.0539 

8 

CONSTRUCT  SENSOR  MONITORING 

PATTERN  (MP) 

-.0523 

9 

COMPENSATE  FOR  PROPAGATION 

CONDITIONS  (PC) 

-.0456 

10 

MANAGE  EQUIPMENT  AND  STORES  (ME) 

.0203 

11 

ADJUST  PATTERN  TO  SENSOR 

FAILURE  (AP) 

.0539 

12 

GAIN  ATTACK  CRITERION  (AC) 

.2025 

13 

DETERMINE  AIRCRAFT  WEAPON 

LAUNCH  POSITION  (LP) 

.3563 

14 

DETERMINE  WEAPON  AND  SETTING 

FOR  ATTACK  (DS) 

.4245 

These  values  represent  the  prioritization  of  the 
decirsion  functions  in  the  ASW  mission  without  attack  on  the 
submarine.  For  reasons  discussed  in  the  previous  section, 
the  smaller  values  represent  higher  priority  in  Table  XV. 
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V.  DISCUSSION 


A.  RESULTS 

1 .  Summary 

In  this  thesis,  a  methodology  was  developed  that 
prioritized  ASW  decision  functions  according  to  different 
criteria.  However,  to  more  fully  understand  the  results  of 
the  methodology  and  its  components,  the  purpose  of  this 
thesis  should  be  reviewed.  The  objective  of  this  effort  was 
to  examine  ASW  decision  making  in  the  S-3A  through  the  use 
of  mathematical  modeling  techniques  and  to  establish  a 
prioritization  technique  for  the  development  of  decision 
aids  to  assist  decision  making  in  the  system.  An  examination 
of  the  S-3A  ASW  decision  making  included  discussions  of  the 
history  of  ASW,  the  four  and  six  partition  spacing  of  the 
Air  ASW  mission,  and  finally  the  fourteen  decision  function 
partition  of  the  Air  ASW  decision  space.  S-3A  TACCOS  sorted 
and  ranked  the  14  decision  functions  through  the  use  of  the 
mathematical  modeling  techniques  of  Multidimensional  Scaling 
and  Unfolding  Analysis,  a  priority  mapping  of  the  fourteen 
decision  functions  was  produced. 

The  Multidimensional  Scaling  representation  of  the 
decision  functions  was  determined  to  have  three  orthogonal 
dimensions  or  axes  through  a  goodness  of  fit  test.  The 
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dimensions  were  interpreted  as  follows: 


Dimension  One  -  Workload 
Dimension  Two  -  Time  Criticality 
Dimension  Three  -  Complexity 
It  should  be  noted  that  this  interpretation  of 
dimensional  meaning  is  not  unique.  The  interpretations,  to  a 
great  degree,  are  subjective  in  nature  relying  upon  the 
analyst's  understanding  of  the  interrelationships  of  the 
stimulus  set.  However,  the  relative  positions  of  the  stimuli 
on  each  dimension  (Figures  8  through  10)  is  unique  so  the 
interpretations  of  the  dimensions  should  be  roughly 
equivalent  from  analyst  to  analyst. 

The  Unfolding  Analysis  determined  that  the 
importance  rankings  were  best  represented  by  the  Generalized 
Distance  Model  in  terms  of  priority  functions.  Only  two  of 
these  four  rankings  were  utilized  in  the  unfolding  analysis. 
The  criterion  of  importance  was  assumed  to  be  the  most 
important  feature  of  the  priority  mapping,  so  the  criteria 
of  importance  to  the  ASW  mission  with  attack  and  importance 
to  the  ASW  mission  without  attack  were  selected  and  the 
criteria  of  urgency  and  workload  rankings  were  used  to 
validate  the  resulting  Priority  Mapping  algorithm. 

Finally,  the  results  of  the  Multidimensional  Scaling 
program  (MINASSA)  and  the  Unfolding  Analysis  program 
(PREFMAP)  were  used  to  'map'  the  precise  priority  function. 
From  the  two  priority  functions,  a  prioritization  of  the  ASW 
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decision  space  was  established  according  to  both  criteria 
(see  Tables  XIII  and  XV). 

As  a  result,  a  priortization  technique  for  Air  ASW 
decision  functions  was  established  for  the  development  of 
decision  aids  in  the  S-3A.  The  fundamental  question  raised 
by  the  priority  ranking  shown  in  Tables  XIII  and  XV  is  what 
is  the  meaning  of  this  prioritization  of  decision  functions. 


2.  Interpretation  of  Results 

To  better  interpret  the  results  of  the  priority 
mapping  function,  a  relative  plot  of  the  14  decision 
functions  is  given  in  Figure  12  for  the  attack  and  no-attack 
criterion.  The  relative  position  of  the  decision  functions 
on  this  priority  scale  for  each  criterion  yields  an  implicit 
ordering  of  areas  in  which  TACCOs  think  decision  aids  would 
be  useful. 

a.  Gain  Attack  Criteria 


The  Gain  Attack  Criteria  decision  function  was 


ranked  first  in  the  priority  scale  for  the  'attack' 
criterion.  Intuitively,  this  result  is  not  surprising  in  a 
mission  where  the  goal  is  attacking  the  submarine.  The 
TACCOs  interviewed  felt  that  in  a  mission  where  attack  on  a 


hostile  threat  was  the  goal,  if  attack  criteria  was  not 
gained,  the  mission  would  be  a  failure. 

The  implication,  from  the  priority  scale  of  the 
importance  to  the  ASW  mission  with  attack,  is  that  more 
energy  should  be  channeled  into  the  area  of  decision  aids 


IMPORTANCE 

W/  ATTACK  W/OUT  ATTACK 


AP  -  ADJUST  PATTERN  TO  SENSOR  FAILURE 

EP  -  EXTEND  SENSOR  PATTERN 

AM  -  ANTICIPATE  TARGET  MOVEMENT 

MP  -  CONSTRUCT  SENSOR  MONITORING  PATTERN 

OH  -  COORDINATE  HAND-OFF 

VC  -  DETERMINE  SIGNAL  IS  VALID  CONTACT 

C6  -  DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK 

K  -  GAIN  ATTACK  CRITERIA 

TF  -  DETERMINE  TARGET  FIX 

C°  -  CREATE  SENSOR  PATTERN 

ME  -  MANAGE  E^OJ . RM  ENT/ STORES 

CS  -  CLASSIFY  SIGNAL 

PC  -  COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 
LP  -  DETERMINE  AIRCRAFT  WEAPON  IAUNCH  PCISITCN 

Figure  12.  Priority  Score  o£  Decision 
Functions  (S-3A) 
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concerned  with  assisting  the  TACCO  in  determining  when 
attack  criteria  has  been  reached.  Additionally,  Gain  Attack 
Criteria  ranked  highest  in  the  average  rankings  for 
importance  to  ASW  mission  with  attack  (Table  V)  and  Workload 
(Table  VIII)  and  second  highest  in  the  ranking  for  the 
Uregency  (Table  VII) .  This  placement  distinguishes  this 
decision  function  as  time-critical  for  the  TACCO. 

Placement  of  the  Gain  Attack  Criteria  decision 
function  in  the  MDS  solution  space  (Figures  S  through  10) 
also  confirms  this  distinction.  Although  this  decision 
function  is  positioned  near  the  middle  of  dimension  three 
(Complexity) ,  it  is  located  at  the  positive  extreme  of  both 
dimension  one  (Workload)  and  two  (Time  Criticality).  This 
positioning  indicates  a  highly  time  critical  and  work 
intensive  decision  function.  Based  upon  this  analysis,  the 
decision  function  -  Gain  Attack  Criteria  -  represents  an 
area  in  the  Air  ASW  mission  where  implementation  of  a 
decision  aid  would  be  of  great  benefit. 

b.  Classify  Signal  and  Determine  Signal  is  Valid 
Contact 

The  decision  functions  -  Classify  Signal  and 
Determine  Signal  is  Valid  Contact  -  were  ranked  first  and 
second  in  the  priority  scale  for  the  'no  attack'  criterion 
(Figure  12)  and  ranked  second  and  third  on  the  priority 
scale  for  the  'attack'  criterion  (behind  Gain  Attack 
Criteria).  This  position  on  the  high  extreme  of  both 
priority  scales  was  unexpected  however  it  was  clear  that  the 
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TACCOs  felt  these  two  decision  functions  to  be  the  heart  of 


a  successful  ASW  mission.  The  TACCOs  felt  that  the  entire 
mission  was  dependent  on  the  proper  evaluation  of  an 
incoming  signal  (that  could  be  a  hostile  threat)  and 
suggested  that  because  this  evaluation  is  so  important, 
there  is  little  room  for  error  regarding  either  decision 
function . 

The  placement  of  these  two  decision  functions 
(Classify  Signal  and  Determine  Signal  is  Valid  Contact)  on 
the  high  extreme  of  both  priority  scales  suggest  that  these 
two  areas  in  the  Air  ASW  mission  are  most  important  in  the 
accomplishment  of  the  mission.  The  implication  is  that 
decision  aids  should  be  developed  for  both  decision 
functions.  Furthermore,  both  decision  functions  were  ranked 
high  in  the  average  rankings  for  importance  to  mission  with 
attack  and  without  attack.  In  the  average  ranking  for 
Urgency,  both  decision  functions  were  positioned  in  the 
center  of  the  ranking,  indicating  that  Classify  Signal  and 
Determine  Signal  is  Valid  Contact  are  not  critical  in  terms 
of  Urgency.  The  placement  of  the  decision  functions  at  the 
bottom  of  the  average  ranking  for  Workload  indicates  that 
both  decision  functions  do  not  add  to  the  TACCO  Workload.  In 
the  S-3A,  Classify  Signal  and  Determine  Signal  is  Valid 
Contact  are  responsibilities  of  the  TACCO  but  these  decision 
functions  are  normally  delegated  to  the  sensor  operators 
with  inputs  to  the  TACCO. 
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The  two  decision  functions  -  Classify  Signal  and 
Determine  Signal  is  Valid  contact  -  in  the  sample  of  the 
fourteen  lay  at  opposite  extremes  in  two  dimensions  and  in 
the  center  of  the  third  of  the  MDS  space  (Figures  5  through 
10).  In  dimension  one,  Workload,  Classify  Signal  and 
Determine  Signal  is  Valid  Contact  lies  at  the  very  negative 
extreme  representing  light  workload  (Figure  8). 

On  the  second  dimension.  Time  Criticality,  the 
two  decision  functions  are  clustered  at  the  'zero'  point, 
well  away  from  the  other  two  well-defined  groupings 
(Figure  9).  This  result  suggest  that  Classify  Signal  and 
Determine  Signal  is  Valid  Contact  are  not  as  time  critical 
as  the  'attack'  functions  because  the  confidence  in  the 
Contact  Validity  normally  improves  over  time.  However,  both 
decision  functions  are  more  time  critical  than  the  'tactics' 
functions  because  confidence  impro  ;es  only  after  contact  is 
established. 

The  third  dimension,  complexity,  located  both 
decision  functions  in  the  positive  region  indicating  a  high 
degree  of  complexity  (Figure  10).  Many  inputs  into  the 
classification  or  verification  decisions  from  all  sensors 
suggest  that  the  degree  of  complexity  increases  with  the 
number  of  sensors  employed.  Classify  Signal  and  Determine 
Signal  is  Valid  is  a  sine  quad  non. 

The  remaining  eleven  decision  functions  were  not 
addressed  further  because  the  apparent  order  of  the  three 
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discussed  decision  functions  is  not  implicitly  present  in 
the  ordering  of  the  last  eleven  functions.  This  problem  will 
be  discussed  further  in  the  Conclusion  and  Recommendations 
sections . 


3.  Comparision  of  Results  to  Similiar  Work 

This  analysis  has  established  priority  scales  for 
both  attack  and  no  attack  decision  situations.  The  fourteen 
decision  functions  were  ordered  in  importance  by  the  use  of 
Multidimensional  Scaling  combined  with  Unfolding  Analysis. 

Zachary  (1980  a  and  b)  used  the  same  prioritization 
methodology  used  in  this  analysis  in  conjunction  with  P-3C 
TACCO  inputs,  the  results  of  his  analysis  was  very  similar 
to  this  analytical  result.  Although  the  P-3C  is  a  shore 
based  ASW  aircraft  with  a  crew  of  11,  the  function  of  the 
TACCO  is  very  much  like  that  of  the  S-3A  TACCO. 
Consequently,  similiar  results  in  the  prioritization 
analysis  would  not  be  surprising. 

Zachary's  prioritization  for  Importance  to  the  ASW 
Mission  with  Attack  on  the  Submarine  was  very  nearly  the 
same  as  the  S-3A  prioritization.  These  were  minor  inversions 
of  one  or  two  decision  functions  in  the  final  priority  scale 
compar isions ,  however  the  differences  in  the  aircraft  system 
capabilities  suggest  that  this  result  is  reasonable.  The 
comparision  of  decision  function  prioritization  for  mission 
without  attack  on  submarine  provided  a  similiar  result.  The 
priority  scales  were  very  similiar  except  for  the  inversion 
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of  one  or  two  decision  functions  due  to  aircraft  and 
tactical  d i ss imi 1 i a r i t i es  .  Figure  13  shows  Zachary's 
priority  scale  for  both  attack  and  no-attack  for  the  P-3C. 
These  priority  scales  are  presented  in  the  same  format  as 
Figure  12  for  ease  of  comparison. 

Zachary's  analysis  also  used  three  dimensions  as 
representative  of  the  MDS  decision  space  (discussed  in 
Chapter  Four).  Zachary's  interpretation  of  his  three 
dimensions  for  the  P-3C  data  was  similiar  to  this  analysis, 
however,  the  placement  of  each  decision  function  in  the 
decision  space  was  markedly  different.  His  dimensional  plots 
can  be  found  in  Appendix  E.  This  difference  could  be 
contributed  to  sampling  differences,  aircraft  differences, 
or  training  differences  between  S-3A  and  P-3C  TACCOs. 

Zachary's  results  reinforces  the  applicability  of 
the  results  of  this  analysis.  Although  two  analyses  of 
similiar  data  do  not  necessarily  validate  a  new  methodology, 
the  degree  of  agreement  between  results  suggest  further 
application  could  be  of  great  benefit  in  the  area  of  ASW. 

B.  CONCLUSION 

This  thesis  has  established  that  a  workable  mathematical 
technique  exist  for  the  prioritization  of  Air  ASW  decision 
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functions,  specifically  and  a  general  application  could  be 
established  for  any  complex  decision  space.  The 
prioritization  methodology  developed  in  this  thesis 
represents  a  new  approach  to  the  problem  of  assigning 
priorities  to  decision  stimuli.  This  methodology  is  based  on 
two  assumptions: 

1)  the  decision  space  is  a  multidimensional  domain 
where  all  underlying  dimensions  have  some 
potential  relevancy  to  the  final  prioritization. 

2)  inputs  to  the  decision  space  must  be  based  on 
experience,  judgement  and  intuition. 

The  prioritization  methodology.  Priority  Mapping, 
produced  two  siginificant  results.  First,  the  MDS  portion  of 
the  methodology  identified  the  three  dimensions  which 
underlie  the  TACCO  decisions.  Secondly,  the  'marriage'  of 
the  MDS  results  and  the  Unfolding  Analysis  provided  a 
numerical  prioritization  of  the  Air  ASW  decision  space. 

The  Priority  Mapping  algorithm  has  several  interesting 
characteristics.  First,  the  sorting  and  ranking  positions  of 
the  data  collection  is  only  required  once.  Repeated  testing 
or  interviewing  of  the  same  subjects  is  not  necessary  as  in 
many  consensus  techniques  such  as  Delphi.  Ease  of  data 
collection  is  a  definite  advantage  of  this  technique. 
Second,  specific  as  well  as  general  questions  can  be 
answered  by  reweighting  the  parameters  of  the  MDS  portion  of 
the  algorithm  (as  discussed  in  Chapter  IV).  Third,  an 


advantage  of  this  algorithm  is  the  built-in  cross  checks  and 
validations  carried  out  by  the  interaction  of  the  average 
rankings,  MDS  unidemnsional  plots,  and  the  priority  scales. 
In  other  words,  placement  of  a  particular  stimulus  on  the 
priority  scale  should  be  reflected  in  that  stimulus' 
placement  on  the  MDS  plots  and  in  the  average  rankings  of 
the  test  criterion.  Additionally,  the  priority  scale  and 
average  ranking  place  would  have  a  positive  influence  on  the 
analyst's  interpretation  of  the  MDS  dimensions.  Fourth,  the 
Priority  Mapping  algorithm  provides  more  information  about 
the  stimulus  set  than  a  ranking  or  an  MDS  dimensional 
analysis  alone.  To  achieve  similar  results  using  only 
ranking  task  or  sorting  task,  the  subjects  would  have  to 
agree  completely  on  the  placement  of  each  stimuli.  The 
algorithm  eliminates  the  need  for  concensus  by  combining  the 
two  techniques  into  a  predictor  model. 

The  operational  application  of  the  priority  mapping 
algorithm  to  fleet  assets  is  where  the  developmental 
'pay-off'  is  received.  This  technique  has  advantages  that 
unit  commanders  can  appreciate: 

1)  ease  of  data  collection  -  simple  forms  with 
simple  instructions.  Any  sailor  could  administer 
the  interview  with  favorable  results. 

2)  speed  of  data  collection  -  fast  with  little 


confusion.  The  unconstructed  sorting  and  rankings 
for  a  large  stimulus  set  would  take  less  time  than 
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other  similiar  techniques  (i.e.  Pairwise  MDS)  with 
much  smaller  stimulus  sets. 

3)  revisit  requirements  -  one  data  collection  visit 
to  operationsl  units.  The  unit  commander  is  not 
compelled  to  provide  subjects  for  data  collection 
or  verification  time  after  time. 

4)  tangible  results  -  a  priority  scale  of  the 
stimulus  set.  A  priority  scale  of  the  stimulus  set 
is  tangible  results  of  the  data  collection, 
providing  real  answers  to  operational  problems. 

Many  of  the  techniques  discussed  in  Chapter  II  do  not 
exhibit  these  advantages.  Fleet  acceptance  of  problem 
solving  methodologies  is  essential  to  the  development  of  new 
software  and  hardware  to  meet  any  present  or  future  threat. 

This  prioritization  methodology  shown  in  Figure  3  can  be 
adjusted  to  fit  any  complex  decision  space  provided  the 
assumptions  of  dimensional  relevency  and  judgmental  inputs 
are  not  violated.  The  methodology  is  flexible  enough  to  be 
used  by  any  branch  of  the  Armed  Forces  or  any  sector  of 
industry.  Areas  of  time  critical,  work  intensive, 
multi-faceted  decision  situations  are  ideal  targets  for  the 
implementation  of  the  prioritization  methodology.  The 
implementation  of  this  technique  will  identify  decision 
situations  where  decision  aids  would  be  most  beneficial. 
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C.  RECOMMENDATIONS 

The  decision  functions  -  Classify  Signal  and  Determine 
Signal  is  Valid  Contact  -  are  at  the  top  of  the  priority 
scale  of  the  14  decision  functions  defining  the  Air  ASW 
decision  space  for  the  S-3A.  Both  decision  functions  were 
the  highest  priority  in  the  no-attack  mission  and  second  and 
third  highest  priority  in  the  attack  mission.  Therfore,  it 
is  recommended  that  a  decision  aid  be  developed  to  assist 
the  S-3A  TACCO  in  the  execution  of  the  decision  funcitons  - 
Classify  Signal  and  Determine  Signal  is  Valid  Contact. 

Although  the  ASW  decision  space  has  been  prioritized  for 
the  S-3A,  the  explicit  priority  ordering  of  Classify  Signal 
and  Determine  Signal  is  Valid  Contact  across  both  priority 
scales  is  not  readily  apparent  in  the  remaining  decision 
function.  In  other  words,  it  is  difficult  to  assess  which  of 
the  remaining  12  members  of  the  S-3A  ASW  decision  space 
would  be  next  in  priority  across  both  priority  scales. 
However,  an  algorithm  for  combining  the  relative  weights  of 
each  priority  scale  across  all  decision  functions  such  as 
the  Mission  Operability  Assessment  Technique  [Helm  and 
Donnell,  1978]  would  solve  the  problem  by  providing  one 
priority  scale  across  all  criterion. 

Therefore,  it  is  recommended  that  an  algorithm  be 
developed  that  would  combine  priority  scalings  over  all 
relevant  criterion  to  achieve  a  combinational  prioritization 
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of  the  decision  space  that  would  be  more  useful  in  the 
development  of  decision  aids. 
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TEXT  OF  INTRODUCTORY  BRIEFING  GIVEN  TO  S-3A  TA£COS  PRIOR 
TO  MULTIDIMENSIONAL  SCALING  INTERVIEWS 

I  would  like  to  begin  by  thanking  you  all  for  coming 
here  today  to  help  us  in  this  effort.  You've  been  asked  to 
come  here  because  of  your  familiarity  with  ASW  and  ASW 
decision  making.  Before  we  ask  you  some  specific  questions 
about  ASW,  I'd  like  to  begin  by  providing  a  brief  background 
on  what  it  is  that  we  mean  by  decision  aids  and  decision 
aiding,  what  the  overall  structure  of  our  program  is,  where 
we  have  been,  and  why  we  have  come  here  to  ta'lk  to  you.  As 
good  a  place  as  any  to  begin  is  with  the  definition  of  a 
decision  aid. 

Very  simply  put,  a  decision  aid  is  any  kind  of  device 
that  helps  humans  make  better,  more  efficient,  clearer,  and 
faster  decisions.  Now,  obviously,  a  wide  range  of  possible 
things  can  be  considered  decision  aids  -  from  a  pencil  and  a 
sheet  of  paper  which  enable  you  to  do  calculations  to  large 
computer  systems  and  programs.  What  we're  most  interested  in 
are  specific  tools  that  will  enable  you  as  a  TACCO  to 
interact  with  your  on-board  computer  system  to  help  you  the 
kinds  of  decisions  that  you  have  to  make  in  the  course  of  an 
ASW  mission. 

The  Navy's  interest  in  decision  aiding  has  been 
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increasing  significantly  recently  because  of  the  realization 
that  warfare,  in  general,  and  ASW,  in  particular,  is 
becoming  more  automated  and  more  highly  technological.  The 
speed  and  complexity  with  which  decisions  must  be  made  is 
increasing  constantly  to  a  point  where,  in  the  not  too 
distant  future,  you  as  TACCOs ,  will  be  overloaded,  possibly 
beyond  your  capability  to  make  necessary  decisions  within  a 
reasonable  amount  of  time.  Therefore,  the  Navy  is  interested 
in  developing  computerized  systems  of  decision  aids  that 
will  help  you  keep  pace  with  the  increasing  automation 
on-board  your  aircraft.  It  should  be  pointed  out  that  these 
decision  aids  will  not  take  you  out  of  the  decision  making 
process  or  automate  your  functions.  Rather,  they  will 
provide  you  with  better,  more  intelligent  support  from 
machines  and  will  give  you  time  to  do  what  you  do  best  - 
think  and  make  decision.  The  whole  concept  of  decision 
aiding  is  based  on  the  notion  that  the  most  complex,  useful, 
and  important  piece  of  equipment  on  any  platform  is  the 
human-brain.  Humans  are  on-board  to  make  decisions;  but 
computers  and  other  kinds  of  devices  can  assist  by  managing 
information,  making  certain  kinds  of  data  available  at  your 
fingertips,  helping  you  remember  things  (like  the  procedures 
you  must  go  through  to  accomplish  a  specific  function), 
performing  certain  kinds  of  calculations  for  you,  and  so 
forth . 

Decision  aids  can  work  at  a  variety  of  levels.  Simple 
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decision  aids  can,  for  example,  provide  very  rudimentary 
bookkeeping  functions  or  provide  checklists  of  things  that 
must  be  done.  More  complex  decision  aids  can  "ask"  you  the 
questions  that  you  should  be  asking  yourself.  Still  more 
complex  decision  aids  can  anticipate  some  of  the  information 
that  you  may  want,  based  on  your  past  performance;  they  can 
perform  certain  kinds  of  calculations  automatically  so  that 
the  results  of  the  calculations  will  be  available  when 
requested;  they  can  make  certain  kinds  of  inferences  about 
what  might  happen  in  the  future  enabling  you  to  play  "what 
if"  games  to  find  out  what  results  might  be  obtained  if  a 
specific  course  of  action  is  undertaken.  These  latter 
things,  which  are  at  the  higher  end  of  the  spectrum  of  the 
capabilities  of  decision  aids,  are  the  ones  that  we're  most 
interested  in. 

To  give  you  a  feeling  for  the  types  of  aids  that  are 
possible,  I'd  like  to  review  some  of  the  other  decision  aid 
projects  of  this  type  that  are  on  going  today  in  the  Navy. 
The  largest  decision  aiding  effort  to  date  has  been 
undertaken  by  the  Office  of  Naval  Research  (ONR) .  Their 
program  has  concentrated  on  developing  a  variety  of  decision 
aids  for  carrier-based  air  strike  operations.  Some  of  the 
specific  problems  to  be  addressed  by  these  decision  aids 
are;  planning  the  ingress  route  for  an  incoming 
carrier-based  air  strike  or  reconnaissance  mission  through  a 
complex  sensor  field;  determining  the  specific  timing  of 


96 


alpha  type  air  strikes;  task  force  EMCOM  planning;  and 
overall  air  strike  campaign  planning.  The  decision  aids 

V 

being  developed  by  QNR  are  very  high  level  decision  aids, 
both  in  the  sense  that  they  provide  a  great  deal  of 
assistance  to  the  human  and  in  the  sense  that  they  deal  with 
very  high  level  command  and  control  decisions  (at  the  task 
force  commander  level  or  higher) . 

More  germaine  to  our  discussion  today  is  the  effort  that 
we  have  undertaken  under  the  joint  sponsorship  of  the  Naval 
Air  Development  Center  and  the  Office  of  Naval  Research  to 
identify  possible  decision  aids  for  on-platform  ASW 
operations.  I'd  like  to  review  some  of  the  earlier  parts  of 
this  effort  to  clarify  why  we  are  here  today. 

We  began  our  effort  by  looking  at  the  ASW  platforms  that 
will  be  in  use  in  the  1980-1985  timeframe  -  the  P-3C,  the 
S-3A,  and  the  LAMPS  MK-III.  We  examined  the  specific 
missions  that  are  undertaken  by  these  three  platforms  in 
order  to  define  a  generic  or  generalized  ASW  mission,  to 
identify  commonalities  in  the  missions  flown  by  the  three 
platforms,  and  to  identify  some  of  the  critical  difference 
both  in  crew  functions  and  in  the  details  of  the  missions 
that  were  undertaken.  We  then  constructed  a  flowchart  of  the 
sequence  of  operations  that  takes  place  in  the  generalized 
mission.  The  mission  was  subdivided  into  very  broad 
categories  -  movement  to  the  search  area,  on-station  search, 
prosecution  of  the  contract,  possibly  culminating  in  attack 
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and  destruction  of  a  hostile  submarine.  These  states  were 
then  subdivided  into  the  detailed  steps  that  involved 
specific  decision  about  classification  and  sensor  extension 
of  search  area,  and  anticipation  of  target  movement.  We 
identified  the  specific  sequences  in  which  these  decisions 
were  made,  recognizing  that  the  interrelations  between  these 
decisions  are  highly  dependent  on  the  sequential  nature  of 
the  ASW  mission.  The  decisions  that  take  place  in  the  attack 
phase,  for  example,  are  dependent  upon  the  succesful 
completion  of  those  portions  of  the  mission  that  relate  to 
search  and  early  prosecution  of  a  contact.  Ultimately,  we 
identified  six  broad  areas  that  we  termed  decision  making 
situations.  We  defined  a  decision  making  situation  as  a 
portion  of  a  mission  in  which  complex  sequence  of  decisions 
has  to  be  made  by  the  TACCO.  These  situations  were  complex 
because  they  involved  trading  off  a  number  of  factors 
against  one  another.  The  six  decision  situations  were: 

1)  On-Station  Search, 

2)  Contact  Classification/Verification 

3)  Target  Localization, 

4)  Surveillance  Tracking, 

5)  Lost  Contact  Reacquisition,  and 

6)  Attack  Planning. 

The  next  issue  we  addressed  in  our  program  was  the 
relative  priority  of  each  of  these  situations.  By 
prioritizing  the  decision  situations,  we  felt  that  we  were 
immediately  confronted  by  the  problem  that  there  was  no 
single  dimension,  or  criteria,  by  which  we  could  prioritize 
the  decision  situations.  They  obviously  are  influenced  by 
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their  sequence  in  the  mission.  They  are  also  influenced  by 
varying  time  constraints  on  the  decisions  that  must  be  made 
in  each  situation  and  how  busy  the  TACCO  is  in  each 
situation.  The  more  we  thought  about  it,  the  more  different 
criteria  for  prioritization  we  were  able  to  define.  It 
became  clear  to  us  that  one  of  the  problems  was  that  we,  as 
analysts,  could  not  determine  the  prioritization.  We  decided 
that  the  only  way  to  determine  the  specific  criteria  that 
were  relevant  to  the  prioritization  of  these  decisions  was 
to  ask  the  people  who  made  these  kinds  of  decisions,  people 
such  as  yourselves.  That  is  why  we  are  here  today.  We  want 
to  determine  how  the  various  decisions  that  you,  as  TACCOs 
make,  should  be  pritoritized.  To  do  this,  we  have  to 
determine  the  dimensions  or  criteria  by  which  these 
decisions  are  interrelated  in  your  minds.  Then  we  have  to 
determine  the  importance  or  priority  of  these  decisions  in  a 
mission. 

There  are  a  number  of  techniques  that  can  be  used  to  do 
this.  One  way  would  be  to  ask  you,  in  very  lengthy  and 
detailed  discussion,  to  try  to  identify  the  dimensions  which 
are  salient  to  ASW  decision  making  for  you.  But  besides 
taking  a  lot  of  your  time,  it  is  not  clear  that  the 
technique  would  work.  People  are  often  very  unclear  about 
the  underlying  principles  they  use  to  think  about  common, 
everyday  things,  like  decision  making.  In  addition,  we 
would  have  the  problem  of  resolving  the  differences  we 
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encountered  between  the  various  people  we  talked  to.  So 
instead,  we  have  decided  to  use  a  more  formal  mathematical 
technique  known  as  multidimensional  scaling  which  will  take 
less  of  your  time  and  will  enable  us  to  determine  both  the 
dimensions  and  the  relative  importance  of  the  various 
decisions  from  the  same  set  of  data.  Multidimensional 
scaling  uses  the  computer  program  to  calculate  the 
dimensions  from  very  simple  judgements  made  by  you  about  the 
basic  similarity  or  difference  among  these  various 
decisions.  We  also  decided  that  we  wanted  to  address  not 
just  the  broad  analytic  catefories  that  we  have  called 
decision  situations,  but  more  precise,  meaningful,  specific 
decisions  that  are  made  by  TACCOs.  We  identified  14  of  these 
decisions,  many  of  which  occur  in  several  of  the  decision 
situations.  There  are,  of  course,  many  more  decisions  that 
are  made  in  the  course  if  a  mission  but  the  14  decisions 
that  we  chose  were  ones  that  appeared  in  more  than  a  single 
decision  situation  or  ones  that  seemed  particulary  amenable 
to  decision  aiding. 

You  have  in  front  of  you  a  set  of  cards. *  Each  card 
describes  one  of  these  decisions.  We're  going  to  ask  you  to 
make  judgements  about  which  decisions  you  feel  are  similar 
or  dissimilar,  and  to  rank  them  by  various  criteria.  The 
results  of  these  judged  similiar ities  and  rankings  will  be 

^Card-packs  describing  the  14  decision  functions  passed  out 
to  interviewees  at  this  point. 
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used  by  the  multidimensional  scaling  process  to 
mathematically  determine  a  set  of  relationships  between 
these  decisions  that  will  help  us  in  prioritizing  them.  It 
will  also  help  us  to  understand  the  kinds  of  distinctions 
that  you  find  most  relevent  among  these  various  decisions. 
But/  most  importantly#  we  feel  it  will  enable  us  to  relate 
our  analysis  in  a  concrete  way  to  your  knowledge# 
experience#  and  intuition  of  the  ASW  missions.  With  your 
help#  we  will  be  able  to  determine  the  best  places  to  apply 
decision  aiding  techniques  in  the  ASW  mission. 

I'd  like  to  add  one  other  note  about  the  way  in  which 
you  should  sort  and  rank  these  decision  functions  in  later 
portions  of  this  procedure.  Do  not  sort  or  rank  the 
decisions  in  terras  of  how  you  think  a  decision  aid  could 
help  you  make  these  decision  nor  in  terms  of  how  you  think 
they  could  be  made  better,  but  rather  in  terms  of  how  you 
currently  go  about  making  these  decisions  and  how  they 
currently  are  handled  on  the  platforms  on  which  you  have 
worked. 
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APPENDIX  B 


RESPONSE  FORMS  FOR  INTERVIEWS 

This  appendix  contains  the  forms  on  which  the  TACCO's 
interviewed  at  NAS  Cecil  Field  recorded  the  results  of  the 
unconstrained  sorting  and  four  rankings.  All  of  the  pages  in 
the  appendix  were  given  to  each  interviewee  as  a  stapled 
packet.  The  first  page  was  used  to  record  some  general 
biographical  information  on  the  respondent.  The  next  two 
pages  provided  the  instructions  and  response  sheet  for  the 
unconstrained  sorting  of  the  ASW  decisions.  The  remaining 
pages  provided  the  instructions  and  response  forms  for  four 
different  rankings  of  the  fourteen  decisions. 
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biographical  information 

Aircraft  Type:  _ 

Rank/Designator:  _ 

Organization: 

Date  TACCO  Designation:  _ 

Date  Mission  Commander  (if  applicable) : 
Deployment  Locations  as  TACCO  (and  dates) : 


INSTRUCTIONS  FOR  THE  'SORTING"  TASK 


You  have  been  given  a  pack  of  fourteen  yellow  caards.  On 
each  of  these  cards  is  a  decision  or  problem  that  is 
encountered  in  an  AS W  mission.  You  have  all  probably  faced 
these  problems  many  times  in  your  experience  as  TACCOs.  Each 
decision  is  in  some  way  different  from  all  the  others,  but 
each  decision  is  not  totally  unique;  some  of  the  decisions 
are  more  alike  than  others.  What  we  would  like  you  to  do  is 
arrange  these  decisions  and  problems  into  groups  according 
to  how  similar  they  are.  That  is,  if  these  are  a  number  of 
cards  which  represent  decisions  or  problems  you  feel  that, 
based  on  your  experience,  are  similar,  then  place  all  these 
cards  together.  If  there  is  a  card  which  you  think  is 
sufficiently  unique  that  it  isn't  similar  to  any  of  the 
others,  then  place  it  in  a  group  by  itself.  There  is  no 
limit  on  the  number  of  groups  you  can  make  or  on  the  number 
of  cards  you  can  place  in  each  group.  While  the  final 
definition  of  what  constitutes  similar  decisions  is  left  to 
you,  we  would  like  you  to  think  of  it  as  referring  to 
decisions  or  problems  which  somehow  solve  in  the  same  way  or 
which  place  similar  demands  on  you  as  TACCO. 
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INSTRUCTIONS:  Record  each  group  of  decisions  you  have  formed 
on  a  separate  block  below.  Take  one  group  and  write  the  code 
for  the  decisions  in  it  on  the  blank  line  provided.  Then  do 
the  same  for  another  group  in  the  next  block  until  each 
group  has  been  recorded  in  a  separate  block.  If  you  wish, 
you  may  also  include  a  short  phrase  describing  the 
similarity  you  saw  in  that  group  of  decisions. 


INSTRUCTIONS  FOR  RANKING  1 


We  would  like  you  to  rank  these  decision  problems  in  the 
order  in  which  a  less-than-optimal  decision  would  have  the 
most  detrimental  effect  on  the  mission.  The  decision  problem 
for  which  a  less-than-optimal  decision  would  have  the  least 
detrimental  impact  on  the  mission  should  be  ranked  last,  and 
the  decision  problem  for  which  a  less-than-optimal  decision 
would  have  the  greatest  detrimental  impact  on  the  mission 
should  be  ranked  first.  For  the  mission,  assume  that  it  is 
an  ASW  mission  in  which  the  submarine  is  to  be  attacked  and 
destroyed  if  possible. 
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INSTRUCTIONS :  Enter  the  two-letter  codes  for  the  decisions 
below  in  the  order  in  which  you  ranked  them.  That  is,  write, 
the  code  for  the  decision  you  ranked  first  next  to  * 1.',  for 
the  decision  you  ranked  second  next  to  '2.'  and  so  on. 
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INSTRUCTIONS  FOR  RANKING  2 

Rank  the  decisions  according  to  the  same 
the  previous  case,  but  for  the  mission, 
mission  in  which  the  submarine  is  only  to 
handed-off  to  a  relief  platform. 


10 


criterion  as  in 
assume  as  ASW 
be  tracked  and 


8 


INSTRUCTIONS ;  Enter  the  two-letter  codes  for  the  decisions 
below  in  the  order  in  which  you  ranked  them.  That  is,  write 


the  code  for  the  decision  you  ranked  first  next  to  *1.', 
for  the  decision  you  ranked  second  next  to  '2.'  and  so  on. 
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INSTRUCTIONS  FOR  RANKING  3 


Rank  the  decisions  according  to  their  urgency.  Define 
urgency  as  referring  to  the  speed  with  which  the  decision 
has  to  be  made  once  you  know  it  must  be  made. 


INSTRUCTIONS:  Enter  the  two-letter  codes  fir  the  decisions 


below  in  the  order  in  which  you  ranked  them.  That  is,  write 
the  code  for  the  decision  you  ranked  first  next  to  '1.',  for 
the  decision  you  ranked  second  next  to  '2.'  and  so  on. 
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INSTRUCTIONS  FOR  RANKING  4 


Rank  these  decisions  according  to  your  workload  during 
each  of  them.  Consider  both  your  cognitive,  or  mental 
workload  and  your  physical  workload  in  ranking  the 
deicisons.  Rank  the  decision  during  which  your  workload  is 
heaviest  first,  and  the  one  during  which  your  workload  is 
least  heavy  last. 
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INSTRUCTIONS:  Enter  the  two-letter  codes  fir  the  decisions 
below  in  the  order  in  which  you  ranked  them.  That  is,  write 
the  code  for  the  decision  you  ranked  first  next  to  *1.',  for 
the  decision  you  ranked  second  next  to  *2.'  and  so  on. 
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APPENDIX  C 


RESULTS  OF  UNCONSTRAINED  SORTINGS  OF  ASW  DECISIONS 

This  appendix  presents  the  results  of  unconstrained 
sortings  of  fourteen  air  ASW  decision  by  30  S-3A  Tactical 
Coordinators  (TACCO's)  stationed  at  NAS  Cecil  Field, 
Florida.  The  sortings  were  performed  as  part  of  interviews 
conducted  between  10  December  1979  and  13  December  1979.  The 
interview  procedure  is  described  in  Chapter  3. 

The  fourteen  decisions  that  were  sorted  are  shown  in 
Table  XVI.  The  instructions  for  the  sorting  task,  and  a 
sample  of  the  form  on  which  the  results  were  recorded,  are 
given  in  Appendix  B. 

Each  sorting  presented  as  several  lists  of  decisions.  In 
this  task,  each  decision  function  is  identified  only  by  a 
two-letter  code.  The  code  used  for  each  decision  is  also 
given  in  Table  XVI. 
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Table  XVI.  Fourteen  ASW  Decision  Function 
and  Alphabetic  Codes 


CODE 

DECISION  FUNCTION 

AP 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

EP 

EXTEND  SENSOR  PATTERN 

AM 

ANTICIPATE  TARGET  MOVEMENT 

MP 

CONSTRUCT  SENSOR  MONITORING  PATTERN 

CH 

COORDINATE  HAND-OFF 

VC 

DETERMINE  SIGNAL  IS  VALID  CONTACT 

DS 

DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK 

AC 

GAIN  ATTACK  CRITERIA 

TF 

DETERMINE  TARGET  FIX 

CP 

CREATE  SENSOR  PATTERN 

ME 

MANAGE  EQUIPMENT/STORES  TO  ACCOMODATE 
PRESENT  AND  FUTURE  NEEDS 

CS 

CLASSIFY  SIGNAL 

PC 

COMPENSATE  FOR  IN  SITU  ACOUSTICAL 

AND  ATMOSPHERIC  PROPAGATION  CONDITIONS 

LP 

DETERMINE  AIRCRAFT  LAUNCH  POSITION  FOR 
ATTACK  ON  TARGET 

SUBJECT  1 

GROUP 

DECISION  FUNCTIONS 

IN 

GROUP 

1 

ME 

2 

CP#MPf PC,AP,EP 

3 

VC,CS 

4 

TF, AM, AC 

5 

DS,  LP 

6 

CH 

SUBJECT  2 

GROUP 

DECISION  FUNCTIONS 

IN 

GROUP 

1 

CP, PC, MP, ME 

2 

EP,  AP 

3 

VC,CS 

4 

TF,  AM 

5 

AC, LP, DS 

6 

CH 
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SUBJECT  3 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP,ME,AP,Mp 

2  EP,  PC 

3  VC,CS 

4  TF,  AM 

5  AC, DS, LP 

6  CH 


SUBJECT  4 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP,MP, EP 

2  PC,ME, AP 

3  VC,CS,TF 

4  AM 

5  DS, LP, AC 

6  CH 


SUBJECT  5 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  AM,CP,MP 

2  PC , AP, ME , EP 

3  VC,CS, TF 

4  AC ,  DS ,  LP 

5  CH 


SUBJECT  6 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

-  cp>pCfMpf Ep 

2  ME, AP 

3  VC,CS,TF 

4  CH 

5  AM ,  AC ,  DS ,  LP 


SUBJECT  7 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  AM,CP,MP, EP, DS 

2  PC 

3  CS,TF,AP 

4  ME 

5  VC 

6  AC , LP 

7  CH 
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1  cs,vc 

2  AC,DS,LP 

3  MP,PC,ME,AP 

4  TF,  AM 

5  CH 

6  CP,  EP 
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SUBJECT  13 
GROUP 


_ DECISION  FUNCTIONS  IN  GROUP 

1  CS,VC 

2  DS,LP,AM,TF,AC 

3  CH 

4  AP, EP,MP, PC  ,CP 

5  ME 


SUBJECT  14 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  AC,AM,DS,TF 

2  CP,EP,AP 

3  MP,LP 

4  VC, PC 

5  CS 

6  ME 

7  CH 


SUBJECT  15 

GROUP  DECISION  FUNCTION  IN  GROUP 

1  CP, PC ,MP,ME,EP,AP 

2  LP , AM , TF , AC 

3  CS,VC 

4  DS 

5  CH 


SUBJECT  16 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  PC,CP,AP,MP,EP 

2  LP,  DS ,  AC ,  AM 

3  CS,VC,TF 

4  ME 

5  CH 


SUBJECT  17 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  ME , AP, DS ,CH 

2  PC, CP, AM, TF, AC 

3  MP,EP 

4  CS,VC 

5  CH 
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SUBJECT  18 
GROUP 


_ DECISION  FUNCTIONS  IN  GROUP 

1  AP,EP,MP,PC,CP 

2  DS / LP , AM , TF , AC 

3  CS,VC 

4  CH 

5  ME 


SUBJECT  19 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  cp,p“c,mp,eR 

2  AM,AC,DS,LP 

3  ME/AP 

4  VC,CS,TF 

5  CH 


SUBJECT  20 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  cp/ep 

2  MP,PC,ME,AP 

3  AC  r  DS ,  LP 

4  TF,AM 

5  CS,VC 

6  CH 


SUBJECT  21 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP,MP,EP,PC,AP 

2  DS  t AC , LP 

3  TF,AM 

4  ME 

5  VC,CS 

6  CH 


SUBJECT  22 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  PC,ME,AP 

2  CP,MP,EP 

3  VC,CS,TF 

4  DS , LP, AC 

5  AM 

6  CH 
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SUBJECT  23 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  AC,DS,LP,AM,TF 

2  MP,AP,ME 

3  PC,VC,CS 

4  CP,  EP 

5  CH 


SUBJECT  24 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  AC,DS,LP 

2  CP,ME,AP,MP 

3  EP, PC 

4  TF, AM 

5  VC,CS 

6  CH 


SUBJECT  25 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  PC,MP,CP,EP,AP 

2  LP, AC, DS 

3  ME 

4  VC,CS 

5  AM, TF 

6  CH 


SUBJECT  26 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  ME, MP, PC, CP 

2  AC ,  LP ,  DS 

3  EP, AP 

4  VC,CS 

5  TF, AM 

6  CH 


SUBJECT  27 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  AP, EP,MP, PC , ME ,dP 

2  AC, LP, AM, TF 

3  VC,CS 

4  DS 

5  CH 
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SUBJECT  28 


GROUP 

DECISION  FUNCTIONS 

IN  GROUP 

1 

CP,MP,PC,EP,AP 

2 

TF/ AM, AC 

3 

ME 

4 

CS,VC 

5 

LP,DS 

6 

CH 

SUBJECT  29 

GROUP 

DECISION  FUNCTIONS 

IN  GROUP 

1 

CP,ME,F,P,TF,AM,AP, 

lp,mF 

2 

VC,CS 

3 

PC 

4 

DS 

5 

AC 

6 

CH 

SUBJECT  30 

GROUP 

DECISION  FUNCTIONS 

IN  GROUP 

1 

CP,ME,AP,MP 

2 

DS, LP, AC 

3 

VC,CS 

4 

TF,  AM 

5 

PC ,  EP 

6 

CH 
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APPENDIX  D 


RESULTS  OF  RANKINGS  OF  ASW  DECISIONS 


This 

append i x 

presents 

the  results  of 

rankings 

of 

fourteen 

Air  ASW 

decisions 

(listed  in  Table 

XVI)  by 

30 

TACCOs  from  NAS  Cecil  Field.  These  rankings  were  performed 
as  part  of  the  interview  described  in  Chapter  3.  The 
fourteen  decisions  were  ranked  according  to  four  different 
criteria.  The  instructions  for  these  rankings  and  the  forms 
on  which  the  results  were  recorded  are  given  in  Appendix  B. 
The  first  ranking  was  done  according  to  the  perceived 
'importance'  of  the  decision  in  a  mission  where  the 
objective  is  to  'attack'  and  destroy  the  hostile  submarine. 
The  second  ranking  was  done  according  to  the  perceived 
'importance'  of  the  decision  function  in  a  mission  where  the 
objective  is  to  survey  the  submarine  only.  The  third  rankign 
was  done  according  to  the  preceived  'urgency'  of  the 
decisions  in  whatever  type  of  mission  gave  then  the  greatest 
urgency.  The  fourth  ranking  was  done  according  to  the 
TACCO's  perceived  'workload'  (both  cognitive  and  physical) 
during  each  of  the  fourteen  decision  functions. 

The  results  of  these  four  rankings  are  given  below.  The 
ranking  generates  by  each  individual  using  each  criteria  is 
presented  as  a  list  of  the  two-letter  codes  used  in  the 


interviews  to  represent  the  decision  functions.  Table  XVI 
(Appendix  C)  contains  the  full  decision  name  represented  by 
each  two-letter  code.  The  order  in  which  the  individuals  are 
listed  is  arbitrary,  but  is  the  same  for  all  four  rankings. 
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RANKING  1 


CRITERION:  IMPORTANCE  TO  MISSION  WITH  ATTACK  ON  SUBMARINE 


SUBJECT  DECISION  FUNCTION  (RANKING  GIVEN) 

NUMBER 


AP 

EP 

AM 

MP 

CH 

VC 

DS 

AC 

TF 

CP 

ME 

CS 

PC 

CP 

1 

7 

8 

2 

5 

13 

9 

11 

3 

1 

4 

14 

10 

6 

12 

2 

13 

10 

4 

11 

6 

8 

2 

1 

5 

14 

9 

7 

12 

3 

3 

6 

5 

10 

3 

14 

7 

13 

11 

9 

1 

4 

8 

2 

12 

4 

12 

13 

7 

8 

14 

4 

2 

1 

6 

10 

11 

5 

9 

3 

5 

9 

12 

8 

13 

14 

2 

4 

1 

7 

11 

10 

3 

6 

5 

6 

11 

12 

1 

4 

14 

5 

9 

8 

7 

3 

13 

6 

2 

10 

7 

11 

6 

2 

7 

13 

8 

4 

1 

5 

10 

14 

9 

12 

3 

8 

12 

6 

2 

3 

14 

8 

4 

10 

9 

1 

13 

7 

5 

11 

9 

13 

12 

5 

11 

14 

10 

4 

1 

3 

7 

8 

9 

6 

2 

10 

10 

13 

6 

12 

14 

2 

11 

5 

4 

3 

7 

1 

8 

9 

11 

9 

12 

15 

8 

13 

7 

3 

2 

4 

11 

14 

6 

10 

1 

12 

8 

11 

6 

12 

13 

10 

7 

4 

3 

2 

9 

1 

14 

5 

13 

13 

9 

6 

10 

14 

12 

3 

4 

5 

7 

11 

1 

8 

2 

14 

13 

10 

6 

9 

14 

2 

4 

8 

7 

1 

12 

3 

11 

5 

15 

9 

11 

5 

3 

13 

4 

7 

8 

6 

2 

14 

1 

12 

10 

16 

11 

12 

6 

9 

14 

8 

4 

2 

3 

7 

13 

5 

10 

1 

17 

9 

7 

8 

10 

14 

13 

11 

1 

5 

3 

6 

12 

4 

2 

18 

11 

9 

10 

5 

13 

12 

4 

3 

6 

1 

14 

8 

7 

2 

19 

9 

11 

7 

8 

14 

6 

2 

4 

5 

12 

13 

1 

10 

3 

20 

12 

11 

7 

9 

14 

1 

5 

4 

3 

10 

13 

2 

8 

6 

21 

11 

12 

4 

10 

14 

2 

8 

3 

7 

5 

13 

1 

6 

9 

22 

12 

13 

8 

10 

14 

2 

6 

5 

4 

9 

1 

3 

11 

7 

23 

13 

12 

8 

10 

14 

4 

6 

2 

5 

7 

11 

3 

9 

1 

24 

14 

10 

5 

12 

6 

2 

8 

4 

3 

11 

13 

1 

7 

9 

25 

10 

11 

8 

9 

14 

13 

3 

1 

7 

5 

4 

6 

12 

2 

26 

6 

7 

12 

10 

14 

13 

3 

2 

9 

5 

8 

1 

11 

4 

27 

11 

12 

13 

3 

14 

6 

9 

8 

7 

2 

4 

5 

1 

10 

28 

12 

10 

7 

11 

4 

2 

9 

3 

6 

13 

8 

1 

14 

5 

29 

12 

7 

4 

14 

9 

1 

6 

3 

5 

11 

13 

12 

10 

8 

30 

8 

7 

4 

11 

5 

14 

2 

3 

6 

10 

9 

13 

12 

1 
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RANKING  2 

CRITERION:  IMPORTANCE  TO  MISSION  WITH  TRACKING  OF  SUBMARINE 

BUT  NO  ATTACK 


SUBJECT  DECISION  FUNCTION  (RANKING  GIVEN) 

NUMBER 


AP 

EP 

AM 

MP 

CH 

VC 

DS 

AC 

TF 

CP 

ME 

CS 

PC 

CP 

1 

4 

5 

9 

2 

10 

6 

14 

12 

8 

1 

11 

7 

3 

13 

2 

11 

8 

2 

9 

4 

6 

14 

1 

3 

12 

7 

5 

10 

13 

3 

6 

5 

10 

3 

11 

7 

14 

12 

9 

1 

4 

8 

2 

13 

4 

9 

10 

2 

8 

3 

4 

13 

12 

1 

7 

11 

6 

5 

14 

5 

7 

10 

5 

11 

6 

1 

12 

14 

4 

9 

8 

2 

3 

13 

6 

9 

8 

1 

4 

10 

5 

13 

12 

7 

3 

11 

6 

2 

14 

7 

7 

8 

11 

2 

4 

5 

14 

12 

9 

1 

10 

6 

3 

13 

8 

11 

5 

2 

3 

9 

7 

13 

14 

8 

1 

10 

6 

4 

12 

9 

11 

8 

2 

9 

7 

6 

14 

13 

1 

10 

3 

5 

4 

12 

10 

9 

11 

8 

10 

5 

2 

14 

12 

4 

3 

6 

1 

7 

13 

11 

7 

10 

2 

6 

1 

5 

13 

12 

3 

9 

11 

4 

8 

14 

12 

12 

9 

5 

10 

6 

11 

14 

4 

3 

2 

7 

1 

8 

13 

13 

7 

11 

4 

10 

2 

12 

14 

3 

5 

9 

6 

l 

8 

13 

14 

11 

8 

5 

7 

4 

2 

14 

12 

6 

1 

10 

3 

9 

13 

15 

11 

13 

6 

4 

9 

3 

8 

7 

5 

1 

14 

2 

12 

10 

16 

10 

9 

2 

8 

3 

5 

14 

12 

1 

7 

11 

4 

6 

13 

17 

7 

4 

5 

6 

10 

11 

14 

8 

9 

1 

2 

12 

3 

13 

18 

7 

6 

9 

2 

11 

4 

14 

13 

10 

1 

8 

5 

3 

12 

19 

12 

8 

4 

6 

1 

3 

14 

13 

5 

10 

9 

2 

7 

11 

20 

10 

9 

4 

7 

5 

1 

14 

12 

3 

8 

11 

2 

6 

13 

21 

12 

13 

5 

11 

4 

2 

9 

3 

8 

6 

14 

1 

7 

10 

22 

9 

10 

5 

7 

11 

2 

13 

12 

4 

6 

1 

3 

8 

14 

23 

8 

13 

3 

14 

5 

4 

10 

9 

2 

7 

6 

1 

12 

11 

24 

11 

8 

4 

10 

5 

2 

13 

12 

3 

9 

7 

1 

6 

14 

25 

9 

8 

6 

7 

1 

11 

13 

12 

5 

3 

2 

4 

10 

14 

26 

6 

7 

3 

9 

4 

10 

13 

12 

2 

5 

8 
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RANKING  3 

CRITERION:  URGENCY  OP  THE  DECISION  FUNCTION  WITHIN  A  MISSION 


SUBJECT  DECISION  FUNCTION  (RANKING  GIVEN) 

NUMBER 


AP 

EP 

AM 

MP 

CH 

VC 

DS 

AC 

TF 

CP 

ME 

CS 

PC 

LP 

1 

12 

8 

4 

10 

14 

1 

6 

5 

3 

9 

13 

2 

11 

7 

2 

9 

8 

5 

13 

6 

4 

2 

3 

7 

14 

11 

10 

12 

1 

3 

11 

9 

3 

13 

14 

7 

2 

5 

4 

12 

10 

6 

8 

1 

4 

9 

8 

4 

10 

14 

1 

7 

5 

3 

11 

12 

2 

13 

6 

5 

7 

11 

10 

13 

8 

4 

3 

1 

9 

14 

12 

5 

6 

2 

6 

4 

5 

3 

13 

9 

10 

6 

7 

2 

14 

12 

11 

8 

1 

7 

6 

7 

4 

11 

14 

8 

2 

3 

5 

12 

13 

9 

10 

1 

8 

8 

7 

4 

13 

14 

9 

3 

2 

6 

12 

5 

10 

11 

1 

9 

1 

9 

12 

11 

14 

4 

7 

6 

5 

13 

8 

3 

10 

2 

10 

3 

9 

6 

14 

13 

11 

12 

1 

7 

5 

4 

10 

8 

2 

11 

7 

13 

4 

14 

10 

1 

3 

8 

9 

2 

5 

11 

6 

12 

12 

9 

12 

2 

8 

13 

5 

7 

1 

3 

11 

14 

4 

10 

6 

13 

12 

11 

5 

13 

9 

4 

3 

1 

7 

14 

8 

6 

10 

2 

14 

6 

12 

4 

11 

14 

8 

5 

2 

3 

13 

7 

9 

10 

1 

15 

11 

10 

5 

13 

9 

6 

2 

1 

4 

12 

8 

7 

14 

3 

16 

12 

11 

3 

9 

14 

2 

13 

4 

7 

8 

10 

1 

6 

5 

17 

12 

14 

8 

13 

11 

10 

3 

1 

6 

7 

5 

9 

4 

2 

18 

8 

9 

7 

10 

12 

13 

4 

2 

3 

11 

5 

14 

6 

1 

19 

6 

9 

5 

13 

11 

7 

4 

2 

3 

14 

10 

8 

12 

1 

20 

12 

10 

6 

11 

14 

4 

7 

1 

5 

8 

13 

3 

9 

2 

21 

7 

10 

5 

9 

13 

11 

2 

1 

4 

8 

14 

12 

6 

3 

22 

7 

14 

5 

12 

13 

3 

8 

1 

4 

11 

9 

6 

10 

2 

23 

13 

14 
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3 
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12 
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13 
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11 
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11 
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1 
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14 

10 
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13 
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10 

9 

3 

13 

6 

4 

2 

5 

11 

14 

8 

7 

12 

1 

29 

12 

8 

4 

13 

11 

9 

2 

3 

5 

6 

7 

10 

14 
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30 
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11 

12 

13 

10 
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1 

9 
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14 
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3 
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RANKING  4 


CRITERION:  WORKLOAD  OF  TACCO  DURING  DECISION  FUNCTION 


SUBJECT  DECISION  FUNCTION  (RANKING  GIVEN) 

NUMBER 


AP 

EP 

AM 

MP 

CH 

VC 

DS 

AC 

TF 

CP 

ME 

CS 

PC 

LP 

1 

6 

7 

2 

8 

14 

12 

4 

3 

1 

9 

11 

13 

10 

5 

2 

8 

9 

1 

13 

4 

11 

6 

3 

2 

14 
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10 
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3 

10 
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11 

14 

12 
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5 
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PLOTS  ON  DECISION  FUNCTIONS  FROM  THREE  DIMENSIONS 
(FROM  ZACHARY  1980b) 
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Figure  14.  Plots  of  Decision  Functions  on  Three 
Dimensions  (from  Zachary  1980b) 
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